美文网首页
azkaban 中条件调度实例

azkaban 中条件调度实例

作者: 风轻云淡lq | 来源:发表于2019-08-23 16:41 被阅读0次

    如果有一个小时任务,但是需要明天在将昨天的数据运行一次怎么做

    这里需要用到azkaban的condition过滤

    1. 定义条件脚本:write_to_props.sh
    curHour=`date +'%H:%M'`
    echo "{\"param1\": \"$curHour\"}" > $JOB_OUTPUT_PROP_FILE
    

    2.定义job:shengtian_click_ratio.job

    nodes:
     - name: JobA
       type: command
       config:
         command: bash ./write_to_props.sh
     
     - name: shengtian_click_ratio
       type: command
       dependsOn:
         - JobA
       config:
         user.to.proxy: hadoop
         retries: 3
         retry.backoff: 3000
         command: sh /opt/gsdwshell/shengtian_click_ratio.sh
       condition: ${JobA:param1} != "01:20"
    
     - name: shengtian_click_ratio_day
       type: command
       dependsOn:
         - JobA
       config:
         user.to.proxy: hadoop
         retries: 3
         retry.backoff: 3000
         command: sh /opt/gsdwshell/shengtian_click_ratio.sh yesterday
       condition: ${JobA:param1} == "01:20"
    
     - name: JobD
       type: command
       dependsOn:
         - shengtian_click_ratio
         - shengtian_click_ratio_day
       config:
         command: pwd
       condition: one_success
    

    当时间是01:20时,需要昨天的数据,当时间非01:20时,运行当前任务,解决小时和天的

    相关文章

      网友评论

          本文标题:azkaban 中条件调度实例

          本文链接:https://www.haomeiwen.com/subject/neutectx.html