-
yarn命令
用法:
yarn [--config confdir] COMMAND参数 描述 resourcemanager -format-state-store 格式化RMStateStore,只有当历史作业不再需要时，才需要格式化 resourcemanager 启动 ResourceManager。用 -format-state-store for 删除 RMStateStore； 使用 -remove-application-from-state-store <appId> 从RMStateStore中删除应用程序 nodemanager 在每个节点启动 NodeManager timelineserver 启动 timeline server rmadmin 管理工具 version 版本信息 jar <jar> 运行 jar 文件 application 查看应用程序状态，或终止应用程序 applicationattempt 打印应用程序尝试报告 container 打印 Container 尝试报告 node 打印节点尝试报告 queue 打印对列信息，只有一个参数 logs 主要用于收集作业日志，注意要等程序结束才能查看日志 classpath 打印 Hadoop 相关依赖和其他依赖库 daemonlog 针对指定的守护进程获取/设置日志优先级 top 启动集群使用工具 CLASSNAME 启动 CLASSNAME 类 示例:
[root@cdh01:~]# yarn version Hadoop 2.6.0-cdh5.12.1 Subversion http://github.com/cloudera/hadoop -r 520d8b072e666e9f21d645ca6a5219fc37535a52 Compiled by jenkins on 2017-08-24T16:43Z Compiled with protoc 2.5.0 From source with checksum de51bf9693ab9426379a1cd28142cea0 This command was run using /opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hadoop-common-2.6.0-cdh5.12.1.jar -
yarn rmadmin 命令
用法:
Usage: yarn rmadmin COMMAND参数 描述 -refreshQueues 重载队列的ACL、状态和调度器特定的属性，ResourceManager将重新加载mapred-queues配置文件 -refreshNodes [-g [timeout in seconds] -client server] 刷新dfs.hosts和dfs.hosts.exclude配置文件，这种方式无需重启NameNode -refreshSuperUserGroupsConfiguration 刷新用户组的配置 -refreshUserToGroupsMappings 刷新用户到组的映射 -refreshAdminAcls 刷新 ResourceManager 的 ACL 管理 -refreshServiceAcl ResourceManager 重载服务级别的授权文件 -getGroups [username] 获取指定用户所属的组 -help [cmd] 显示指定命令的帮助，没有指定，则显示全部指令帮助 -addToClusterNodeLabels [label1,label2,label3] (label splitted by ",") 添加标签 -removeFromClusterNodeLabels [label1,label2,label3] (label splitted by ",") 移除标签 -replaceLabelsOnNode [label1,label2,label3] (label splitted by ",") 替换标签 -directlyAccessNodeLabelStore -transitionToActive <serviceId> [--forceactive] -transitionToStandby <serviceId> -failover [--forcefence] [--forceactive] <serviceId> <serviceId> -getServiceState <serviceId> -checkHealth <serviceId> 示例:
[root@cdh01:~]# yarn rmadmin -getGroups hdfs hdfs : hdfs hadoop supergroup -
yarn application 命令
用法:
Usage: yarn application COMMAND参数 描述 -appStates <States> 与 -list 配合使用，基于应用程序的状态来过滤,多个状态用逗号分隔状态。有效的状态: ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUNNING,FINISHED,FAILED,KILLED -appTypes <Types> 与 -list 配合使用，基于应用程序的类型来过滤，多个应用程序用逗号分隔 -help 帮助列表 -kill <Application ID> 杀死指定的应用程序 -list 从 ResourceManager 获取应用程序列表. 使用 -appTypes 基于应用程序的类型来过滤 使用 -appStates 基于应用程序的状态来过滤 -movetoqueue <Application ID> 将应用程序移动到不同的队列 -queue <Queue Name> 与 movetoqueue配合使用，移到指定队列 -status <Application ID> 获取应用程序的状态 示例:
[root@cdh01:~]# yarn application -status application_1557221110563_0246 ... Application Report : Application-Id : application_1557221110563_0246 Application-Name : INSERT OVERWRITE TABLE hdsc...hdsc.base_user(Stage-1) Application-Type : MAPREDUCE User : admin Queue : default Start-Time : 1557366149859 Finish-Time : 1557366154321 Progress : 100% State : KILLED Final-State : KILLED Tracking-URL : null RPC Port : -1 AM Host : N/A Aggregate Resource Allocation : 3111 MB-seconds, 2 vcore-seconds Log Aggregation Status : NOT_START Diagnostics : Application killed by user. -
yarn applicationattempt 命令
用法:
Usage: yarn applicationattempt COMMAND参数 描述 -help 帮助列表 -list <Application ID> 获取应用程序尝试的列表 -status <Application Attempt ID> 获取应用程序尝试的状态 示例:
[root@cdh01:~]# yarn applicationattempt -list application_1557221110563_0246 ... Total number of application attempts :1 ApplicationAttempt-Id State AM-Container-Id Tracking-URL appattempt_1557221110563_0246_000001 KILLED container_e52_1557221110563_0246_01_000002 http://slave1:8088/proxy/application_1557221110563_0246/ [root@cdh01:~]# yarn applicationattempt -status appattempt_1557221110563_0246_000001 Application Attempt Report : ApplicationAttempt-Id : appattempt_1557221110563_0246_000001 State : KILLED AMContainer : container_e52_1557221110563_0246_01_000002 Tracking-URL : http://slave1:8088/proxy/application_1557221110563_0246/ RPC Port : -1 AM Host : N/A Diagnostics : Attempt recovered after RM restart -
yarn container 命令
用法:
Usage: yarn container COMMAND参数 描述 -help 帮助列表 -list <Application Attempt ID> 获取 Container 列表 -status <Container ID> 获取 Container 的状态 示例：
[root@cdh01:~]# yarn container -list appattempt_1557221110563_0369_000001 ... Total number of containers :0 Container-Id Start Time Finish Time State Host LOG-URL -
yarn node 命令
用法:
yarn node COMMAND参数 描述 -all 与 -list 打印所有节点的报告 -list 获取所有 RUNNING 状态的节点。 支持 -states 选项过滤指定的状态。节点状态包括 NEW,RUNNING,UNHEALTH,DECOMMISSIONED,LOST 和 REBOOTED -states <States> 与 -list 配合使用，用逗号分隔节点状态，只显示这些状态的节点信息 -status <NodeId> 获取指点节点的状态 示例：
[root@cdh01:~]# yarn node -all -list ... Total Nodes:5 Node-Id Node-State Node-Http-Address Number-of-Running-Containers cdh01:8041 RUNNING cdh01:8042 0 cdh02:8041 RUNNING cdh02:8042 0 cdh03:8041 RUNNING cdh03:8042 0 cdh04:8041 RUNNING cdh04:8042 0 cdh05:8041 RUNNING cdh05:8042 0 [root@cdh01:~]# yarn node -status cdh01:8041 Node Report : Node-Id : cdh01:8041 Rack : /default Node-State : RUNNING Node-Http-Address : cdh01:8042 Last-Health-Update : Thu 30/May/19 08:14:31:270CST Health-Report : Containers : 0 Memory-Used : 0MB Memory-Capacity : 12288MB CPU-Used : 0 vcores CPU-Capacity : 4 vcores Node-Labels : -
yarn queue 命令
用法:
yarn queue COMMAND参数 描述 -help 帮助列表 -status <Queue Name> 获取队列的状态 示例：
[root@cdh01:~]# yarn queue -status default ... Queue Information : Queue Name : root.default State : RUNNING Capacity : .0% Current Capacity : .0% Maximum Capacity : -100.0% Default Node Label expression : Accessible Node Labels : -
yarn logs 命令
用法:
yarn logs -applicationId <application ID> [OPTIONS]参数 描述 -appOwner <Application Owner> 指定应用程序的所有者 -containerId <Container ID> 指定运行任务的 Container ID -nodeAddress <Node Address> 指定运行任务的节点，格式为 host:name -
yarn daemonlog 命令
用法:
yarn daemonlog -getlevel <host:port> <classname> [-protocol (http|https)] 或 yarn daemonlog -setlevel <host:port> <classname> <level> [-protocol (http|https)]参数 描述 -getlevel <host:port> <classname> 针对指定的守护进程获取日志优先级 -setlevel <host:port> <classname> <level> [-protocol (http https) 针对指定的守护进程设置日志优先级 示例:
[root@cdh01:~]# yarn daemonlog -getlevel cdh01:50075 org.apache.hadoop.hdfs.server.datanode.DataNode root@bigdata-sit-54:~# yarn daemonlog -getlevel bigdata-sit-54:50075 org.apache.hadoop.hdfs.server.datanode.DataNode Connecting to http://cdh01:50075/logLevel?log=org.apache.hadoop.hdfs.server.datanode.DataNode Submitted Log Name: org.apache.hadoop.hdfs.server.datanode.DataNode Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO [root@cdh01:~]# yarn daemonlog -getlevel cdh01:8088 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Connecting to http://cdh01:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Submitted Log Name: org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO [root@cdh01:~]# yarn daemonlog -getlevel cdh01:19888 org.apache.hadoop.mapreduce.v2.hs.JobHistory Connecting to http://cdh01:19888/logLevel?log=org.apache.hadoop.mapreduce.v2.hs.JobHistory Submitted Log Name: org.apache.hadoop.mapreduce.v2.hs.JobHistory Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO
网友评论