一. 问题描述

访问 job history的时候，发现一长串描述，但是没有地方查看对应的日志。

Logs not available for container_1668064306391_0004_01_000001. Aggregation may not be complete, Check back later or try to find the container logs in the local directory of nodemanager hp6:43050

image.png

二. 解决方案

2.1 mapred-site.xml已添加如下配置：

<property>
        <name>mapreduce.jobhistory.address</name>
        <value>hadoop000:10020</value>
        <description>MapReduce JobHistory Server IPC host:port</description>
</property>

<property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>hadoop000:19888</value>
        <description>MapReduce JobHistory Server Web UI host:port</description>
</property>

<property>
        <name>mapreduce.jobhistory.done-dir</name>
        <value>/history/done</value>
</property>

<property>
        <name>mapreduce.jobhistory.intermediate-done-dir</name>
        <value>/history/done_intermediate</value>
</property>

2.2 日志聚合功能已开启：

yarn-site.xml

<property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
</property>

2.3 job historyserver 已经配置完毕

[hadoop@hadoop000 hadoop]$ jps | grep -v Jps
37602 ResourceManager
4558 SecondaryNameNode
4271 NameNode
4372 DataNode
36310 JobHistoryServer
37699 NodeManager