美文网首页程序员
AZKABAN(三)curl命令

AZKABAN(三)curl命令

作者: 进击的码奴 | 来源:发表于2018-07-22 20:14 被阅读0次

        因为很多服务器并不能连接外网。那么这时web页面就没法使用了。这时可以使用azkaban接口代码,写自己的交互式命令。也可以直接使用提供的curl命令。

        Azkaban 操作需要登录鉴权。登录成功后返回一个session.id,以后通过session.id进行身份验证。因此每次使用curl指令除了登录操作,其他都需要这个参数。

    本文中$id $p 之类。都是 使用shell变量 保存id,工程id等变量。如有问题可以留言。会一一解答。

    登录

    curl -k -X POST --data "action=login&username=$name&password=$pw"https://localhost:8443

    登录成功返回体

    {

      "status" :"success",

      "session.id" :"c001aba5-a90f-4daf-8f11-62330d034c0a"

    }

    Session.id 后续操作需要重复使用。

    创建工程

    curl -k -X POST --data "session.id=$id&name=sh&description=sh"https://localhost:8443/manager?action=create

           成功返回体

           {

      "status":"success",

     "path":"manager?project=aaaa", //工程创建后路径

                "action":"redirect"

    }

    删除工程

    curl -k --get --data "session.id=session.id&delete=true&project=工程名称"https://localhost:8443/manager

    没有返回

    上传工程

    curl -k -i -H "Content-Type:

    multipart/mixed" -X POST --form “session.id=$id” --form 'ajax=upload'

    --form 'file=@myproject.zip;type=application/zip' --form 'project=sh' https://localhost:8443/manager

    原azkaban doc中工程名后面有 ;type/plain 但是执行出错。去掉执行真确。正确返回为。

    HTTP/1.1 100 Continue

    HTTP/1.1 200 OK

    Content-Type: application/json

    Content-Length: 42

    Server: Jetty(6.1.26)

    {

      "projectId" :"6",

      "version" :"1"

    }

    注意点,上传文件格式为zip。且注意上传路径。

    获取工程流。

    curl -k --get --data "session.id=$id&ajax=fetchprojectflows&project=$p"https://localhost:8443/manager

    正确返回

    {

      "flows" : [ {

        "flowId" :"o2o_2_hive"

      } ],

      "project" :"curl-test",

      "projectId" : 6

    }

    获取工作流

    curl -k --get --data "session.id=$id&ajax=fetchflowgraph&project=$p&flow=$fd"https://localhost:8443/manager

    正确返回:

    {

      "nodes" : [ {

        "in" : ["o2o_clean_data" ],

        "id" :"o2o_2_hive",

        "type" :"command "

      }, {

        "in" : ["o2o_up_2_hdfs" ],

        "id" :"o2o_clean_data",

        "type" :"command "

      }, {

        "id" :"o2o_get_file_ftp1",

        "type" :"command "

      }, {

        "id" :"o2o_get_file_ftp2",

        "type" :"command "

      }, {

        "in" : ["o2o_get_file_ftp1", "o2o_get_file_ftp2" ],

        "id" : "o2o_up_2_hdfs",

        "type" :"command "

      } ],

      "project" :"curl-test",

      "projectId" : 6,

      "flow" :"o2o_2_hive"

    }

    In表示依赖的意思。Id表示工程名。Type表示job类型。

    获取执行流

    curl -k --get --data "session.id=$id&ajax=fetchFlowExecutions&project=$p&flow=$fd&start=0&length=5"https://localhost:8443/manager

    {

      "executions" : [ {

        "startTime" :1407779928865,

        "submitUser" :"1",

        "status" :"FAILED",

        "submitTime" :1407779928829,

        "execId" : 306,

        "projectId" :192,

        "endTime" :1407779950602,

        "flowId" :"test"

      }, {

        "startTime" :1407779877807,

        "submitUser" :"1",

        "status" :"FAILED",

        "submitTime" :1407779877779,

        "execId" : 305,

        "projectId" :192,

        "endTime" :1407779899599,

        "flowId" :"test"

      }, {

        "startTime" :1407779473354,

        "submitUser" :"1",

        "status" :"FAILED",

        "submitTime" :1407779473318,

        "execId" : 304,

        "projectId" : 192,

        "endTime" :1407779495093,

        "flowId" :"test"

      } ],

      "total" : 16,

      "project" : "curl-test",

      "length" : 3,

      "from" : 0,

      "flow" : "o2o_2_hive",

      "projectId" : 192

    }

    上面显示了总共的执行次数,和开始结束时间的等等信息。

    获取运行流。

    curl -k --data "session.id=$id&ajax=getRunning&project=$p&flow=$fd"https://localhost:8443/executor

    成功返回

    {

      "execIds": [301,302]

    }

    执行一个流

    curl -k --get --data'session.id=189b956b-f39f-421e-9a95-e3117e7543c9' --data 'ajax=executeFlow' --data'project=azkaban-test-project' --data 'flow=test'https://localhost:8443/executor

    成功返回

    {

      "project" :"curl-test",

      "message" :"Execution submitted successfully with exec id 10",

      "flow" : "o2o_2_hive",

      "execid" : 10

    }

           具体参数如下:

           ParameterDescription

    session.idThe user session id.

    Example Values:30d538e2-4794-4e7e-8a35-25a9e2fd5300

    ajax=executeFlowThe fixed parameter indicating the current ajax action is  executeFlow.

    projectThe project name of the executing flow.

    Example Values: run-all-jobs

    flowThe flow id to be executed.

    Example Values: test-flow

    disabled (optional)A list of job names that should be disabled for this  execution. Should be formatted as a JSON Array String.

    Example Values: ["job_name_1",  "job_name_2", "job_name_N"]

    successEmails (optional)A list of emails to be notified if the execution succeeds. All  emails are delimitted with [,|;|\\s+].

    Example Values: foo@email.com,bar@email.com

    failureEmails (optional)A list of emails to be notified if the execution fails. All  emails are delimitted with [,|;|\\s+].

    Example Values: foo@email.com,bar@email.com

    successEmailsOverride (optional)Whether uses system default email settings to override  successEmails.

    Possible Values: true,  false

    failureEmailsOverride (optional)Whether uses system default email settings to override  failureEmails.

    Possible Values: true,  false

    notifyFailureFirst (optional)Whether sends out email notifications as long as the first  failure occurs.

    Possible Values: true,  false

    notifyFailureLast (optional)Whether sends out email notifications as long as the last  failure occurs.

    Possible Values: true,  false

    failureAction (Optional)If a failure occurs, how should the execution behaves.

    Possible Values: finishCurrent,  cancelImmediately, finishPossible

    concurrentOption (Optional)Concurrent choices. Use ignore if nothing specifical is  required.

    Possible Values: ignore,  pipeline, skip

    flowOverride[flowProperty] (Optional)Override specified flow property with specified value.

    Example Values : flowOverride[failure.email]=test@gmail.com

    完整参数执行如下:

    curl -k --get --data'session.id=079f7617-1454-4c85-890fd829d1e3b502' --data 'ajax=executeFlow'--data 'project=curl-test' --data 'flow=o2o_2_hive' --data'disabled=["o2o_up_2_hdfs"]' --data'successEmails=15800968375@163.com' --data 'failureEmails=15800968375@163.com'--data 'notifyFailureFirst=true' --data 'failureEmailsOverride=true' --data'failureAction=finishCurrent'  https://localhost:8443/executor

    其中如果想让设置的email发送成功必须emailsoverride设置为true。英文描述有问题,是当前设置覆盖默认设置。描述反了。

    取消一个运行流

    curl -k --data "session.id=session.id&ajax=cancelFlow&execid=execid"https://localhost:8443/executor

    如果没有这个execid那么会报错。

    {

    "error" : "Execution 14 of flow o2o_2_hive isn'trunning."

    }

    如果成功返回{}

    设置定时任务

    curl -k -X POST --data"session.id=$id&ajax=scheduleFlow&projectName=$p&projectId=$pd&flow=$fd&scheduleTime=10,30,am,UTC&scheduleDate=07/02/2018&is_recurring=on&period=1d"https://localhost:8443/schedule

    成功返回

    {

      "message" :"curl-test.o2o_2_hive scheduled.",

      "status" :"success"

    }

    ParameterDescription

    session.idThe  user session id.

    ajax=scheduleFlowThe  fixed parameter indicating the action is to schedule a flow.

    projectNameThe  name of the project.

    projectIdThe

      id of the project. You can find this with Fetch

      Flows of a Project.

    flowThe  name of the flow id.

    scheduleTime(with  timezone)The  time to schedule the flow. Example: 12,00,pm,PDT (Unless UTC is specified,  Azkaban will take current server's default timezone instead)

    scheduleDateThe  date to schedule the flow. Example: 07/22/2014

    is_recurring=on  (optional)Flags  the schedule as a recurring schedule.

    period  (optional)Specifies  the recursion period. Depends on the "is_recurring" flag being set.  Example: 5w

    Possible Values:

    MMonths

    wWeeks

    dDays

    hHours

    mMinutes

    sSeconds

    注:这个官方文档有区别,测试过了官方文档错的。

    使用cron设置定时任务(2.5不支持这是最新版本支持)

    curl -k -X POST --data"session.id=$id&ajax=scheduleCronFlow&projectName=$p&flow=$fd&cronExpression=0031 10 ? ? ?"https://localhost:8443/schedule

    cron定时任务工具:http://cron.qqe2.com/

    查看定时工作

    curl -k --get --data"session.id=$id&ajax=fetchSchedule&projectId=$pd&flowId=$fd"https://localhost:8443/schedule

    成功返回为:

    {

      "schedule" : {

        "nextExecTime" :"2018-07-03 03:31:00",

        "period" :"1 day(s)",

        "submitUser" :"admin",

        "scheduleId" :"9",

        "firstSchedTime": "2018-07-02 03:31:00"

      }

    }

           请求体中projectId为工程id。

    取消一个定时任务

    curl -k -X POST --data"session.id=$id&action=removeSched&scheduleId=9"https://localhost:8443/schedule

    成功为:

    {

      "message" :"flow o2o_2_hive removed from Schedules.",

      "status" :"success"

    }

    设置sla(定时任务执行通知)

    curl -k -X POST --data

    "session.id=$id&ajax=setSla&scheduleId=$sd&slaEmails=15800968375@163.com&settings[0]=aaa,SUCCESS,5:00,true,false"

    https://localhost:8443/schedule

    成功返回:{}

    Setting内具体参数的意思是:第一个是id,第二个规则,第三个时间间隔,第四个emailAction,第五个killAction

    查看sla

    curl -k --get --data "session.id=$id&ajax=slaInfo&scheduleId=$sd"https://localhost:8443/schedule

    成功返回:

    {

      "settings" : [ {

        "duration" :"300m",

        "rule" :"SUCCESS",

        "id" :"aaa",

        "actions" : ["EMAIL" ]

      } ],

      "slaEmails" : ["15800968375@163.com" ],

     "allJobNames" : [ "o2o_up_2_hdfs","o2o_2_hive", "o2o_get_file_ftp1","o2o_clean_data", "o2o_get_file_ftp2" ]

    }

    暂停一个工作流

    curl -k --get --data"session.id=$id&ajax=pauseFlow&execid=3"https://localhost:8443/executor

    如果execid不存在会返回如下:

    {

      "error" :"Execution 3 of flow o2o_2_hive isn't running."

    }

           成功为{}

    重新启动一个工作流

    curl -k  --get --data"session.id=$id&ajax=resumeFlow&execid=3"https://localhost:8443/executor

    如果exeid不存在:

    {

      "resume" :"Execution 3 of flow o2o_2_hive isn't running."

    }

           成功返回{}

    取得执行流运行状况

    curl -k --get --data"session.id=$id&ajax=pauseFlow&execid=3"https://localhost:8443/executor

    如果id不存在:{

      "error" :"Execution 3 of flow o2o_2_hive isn't running."

    }

    成功返回:

    {

      "attempt" : 0,

      "submitUser" :"1",

      "updateTime" :1407779495095,

      "status" :"FAILED",

      "submitTime" :1407779473318,

      "projectId" : 192,

      "flow" :"test",

      "endTime" :1407779495093,

      "type" : null,

      "nestedId" :"test",

      "startTime" :1407779473354,

      "id" :"test",

      "project" :"test-azkaban",

      "nodes" : [ {

        "attempt" : 0,

        "startTime" :1407779495077,

        "id" :"test",

        "updateTime" :1407779495077,

        "status" :"CANCELLED",

        "nestedId" :"test",

        "type" :"command",

        "endTime" :1407779495077,

        "in" : ["test-foo" ]

      }, {

        "attempt" : 0,

        "startTime" :1407779473357,

        "id" :"test-bar",

        "updateTime" :1407779484241,

        "status" :"SUCCEEDED",

        "nestedId" :"test-bar",

        "type" :"pig",

        "endTime" :1407779484236

      }, {

        "attempt" : 0,

        "startTime" :1407779484240,

        "id" :"test-foobar",

        "updateTime" :1407779495073,

        "status" :"FAILED",

        "nestedId" :"test-foobar",

        "type" :"java",

        "endTime" : 1407779495068,

        "in" : ["test-bar" ]

      }, {

        "attempt" : 0,

        "startTime" :1407779495069,

        "id" :"test-foo",

        "updateTime" :1407779495069,

        "status" :"CANCELLED",

        "nestedId" :"test-foo",

        "type" :"java",

        "endTime" :1407779495069,

        "in" : ["test-foobar" ]

      } ],

      "flowId" :"test",

      "execid" : 304

    }

    取得执行日志

    curl -k --get --data"session.id=$id&ajax=fetchExecJobLogs&execid=7&jobId=6&offset=0&length=100"https://localhost:8443/executor

    执行成功:

    {

      "data" :"05-08-2014 16:53:02 PDT test-foobar INFO - Starting job test-foobar at140728278",

      "length" : 100,

      "offset" : 0

    }

           失败:

                  {

                "error" : "Job 6doesn't exist in 7"

    }

    获得执行流更新日志

    curl -k --get  --data"session.id=$id&ajax=fetchexecflowupdateexecid=6&lastUpdateTime=-1"https://localhost:8443/executor

    成功返回:

    {

      "nodes" : [ {

        "startTime" :1530194417779,

        "updateTime" :1530194417779,

        "id" :"o2o_up_2_hdfs",

        "endTime" :1530194417779,

        "attempt" : 0,

        "status" :"CANCELLED"

      }, {

        "startTime" :1530194417871,

        "updateTime" :1530194417871,

        "id" :"o2o_2_hive",

        "endTime" :1530194417871,

        "attempt" : 0,

        "status" :"CANCELLED"

      }, {

        "startTime" :1530194417688,

        "updateTime" :1530194417819,

        "id" :"o2o_get_file_ftp1",

        "endTime" :1530194417773,

        "attempt" : 0,

        "status" :"FAILED"

      }, {

        "startTime" :1530194417826,

        "updateTime" :1530194417826,

        "id" :"o2o_clean_data",

        "endTime" :1530194417826,

        "attempt" : 0,

        "status" :"CANCELLED"

      }, {

        "startTime" :1530194417696,

        "updateTime" :1530194417785,

        "id" :"o2o_get_file_ftp2",

        "endTime" :1530194417727,

        "attempt" : 0,

        "status" : "FAILED"

      } ],

      "startTime" :1530194417642,

      "updateTime" :1530194417999,

      "id" :"o2o_2_hive",

      "endTime" :1530194417985,

      "attempt" : 0,

      "flow" :"o2o_2_hive",

      "status" :"FAILED"

    }

    相关文章

      网友评论

        本文标题:AZKABAN(三)curl命令

        本文链接:https://www.haomeiwen.com/subject/duolmftx.html