美文网首页
yarn容器启动失败解决办法

yarn容器启动失败解决办法

作者: 臻霏 | 来源:发表于2019-06-23 21:10 被阅读0次

    19/06/23 23:21:40 INFO mapreduce.Job: map 0% reduce 0%
    19/06/23 23:21:41 INFO mapreduce.Job: Task Id : attempt_1561298854921_0001_m_000000_0, Status : FAILED
    Container launch failed for container_1561298854921_0001_01_000002 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: slave_0:40671
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:211)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:210)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:196)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:117)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    19/06/23 23:21:43 INFO mapreduce.Job: Task Id : attempt_1561298854921_0001_m_000000_1, Status : FAILED
    Container launch failed for container_1561298854921_0001_01_000003 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: slave_0:40671
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:211)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:210)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:196)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:117)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    19/06/23 23:21:45 INFO mapreduce.Job: Task Id : attempt_1561298854921_0001_m_000000_2, Status : FAILED
    Container launch failed for container_1561298854921_0001_01_000004 : java.lang.IllegalArgumentException: Does not contain a valid host:port authority: slave_0:40671
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:211)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.newProxy(ContainerManagementProtocolProxy.java:210)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy$ContainerManagementProtocolProxyData.<init>(ContainerManagementProtocolProxy.java:196)
    at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy(ContainerManagementProtocolProxy.java:117)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy(ContainerLauncherImpl.java:403)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:138)
    at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:369)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    19/06/23 23:21:48 INFO mapreduce.Job: map 100% reduce 100%
    19/06/23 23:21:49 INFO mapreduce.Job: Job job_1561298854921_0001 failed with state FAILED due to: Task failed task_1561298854921_0001_m_000000
    Job failed as tasks failed. failedMaps:1 failedReduces:0
    19/06/23 23:21:49 INFO mapreduce.Job: Counters: 4
    Job Counters
    Other local map tasks=3
    Data-local map tasks=1
    Total time spent by all maps in occupied slots (ms)=0
    Total time spent by all reduces in occupied slots (ms)=0

    出现这种问题的原因是:YARN容器启动失败

    解决办法

    修改datanode节点的名称不包含特殊字符,比如我原先的datanode节点的名称是slave_0和 slave_1,修改成slave0和slave1后,mapreduce程序可以正常运行。

    相关文章

      网友评论

          本文标题:yarn容器启动失败解决办法

          本文链接:https://www.haomeiwen.com/subject/ynccqctx.html