一. 问题描述
访问 job history的时候,发现一长串描述,但是没有地方查看对应的日志。
Logs not available for container_1668064306391_0004_01_000001. Aggregation may not be complete, Check back later or try to find the container logs in the local directory of nodemanager hp6:43050
image.png
二. 解决方案
2.1 mapred-site.xml已添加如下配置:
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop000:10020</value>
<description>MapReduce JobHistory Server IPC host:port</description>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop000:19888</value>
<description>MapReduce JobHistory Server Web UI host:port</description>
</property>
<property>
<name>mapreduce.jobhistory.done-dir</name>
<value>/history/done</value>
</property>
<property>
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/history/done_intermediate</value>
</property>
2.2 日志聚合功能已开启:
yarn-site.xml
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
2.3 job historyserver 已经配置完毕
[hadoop@hadoop000 hadoop]$ jps | grep -v Jps
37602 ResourceManager
4558 SecondaryNameNode
4271 NameNode
4372 DataNode
36310 JobHistoryServer
37699 NodeManager
网友评论