yarn常见问题总结

1、timelineserver启动问题

1.1、报错描述

org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 3 missing files; e.g.: /hadoop/yarn/timeline/leveldb-timeline-store.ldb/000064.sst
    at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:104)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:172)
    at org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:182)
Caused by: org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 3 missing files; e.g.: /hadoop/yarn/timeline/leveldb-timeline-store.ldb/000064.sst
    at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200)
    at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218)
    at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168)
    at org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore.serviceInit(LeveldbTimelineStore.java:229)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

1.2、解决

删除/hadoop/yarn/timeline/目录的数据重新启动，这个删除会导致yarn历史的application数据丢失，但是内存中的不会掉

1.3、yarn timelineserver启动

yarn timelineserver

2. yarn rm节点 8032端口

yarn rm的8032端口只会在激活的那台机器开启，standby 节点不会有8032端口

3、yarn 超级用户代理问题

3.1、场景描述

使用root用户启动kyuubi，导致Hadoop集群不响应报错

for port 8032: readAndProcess from client 10.111.111.111 threw exception [org.apache.hadoop.security.authorize.AuthorizationException: User: root is not allowed to impersonate 011212111]

3.2、解决

调整kyuubi启动用户为hive

备注说明：链接失败在resourcemanager日志中会打印出来，如果访问失败可以先查日志，如果日志没有可以调整下日志等级