yarn管理界面中的错误日志:
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:604)
at org.apache.hadoop.util.Shell.run(Shell.java:507)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Container exited with a non-zero exit code 1
看到这个问题是去百度了一下又说是路径权限问题的,但并不是!
于是我去服务器中下载了更加完整的日志文件
命令如下:
yarn logs -applicationId application_1542248786737_0626 > yarn.log
打开日志后翻到最下面看到如下内容:
Container: container_1542248786737_0626_02_000003 on cdh-005_8041
===================================================================
LogType:stderr
Log Upload Time:Mon Nov 26 18:56:09 +0800 2018
LogLength:142
Log Contents:
Invalid maximum heap size: -Xmx0m
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
LogType:stdout
Log Upload Time:Mon Nov 26 18:56:09 +0800 2018
LogLength:0
Log Contents:
看到下面的内容小伙伴们是不是知道是为什么了
Invalid maximum heap size: -Xmx0m
Error: Could not create the Java Virtual Machine.
内存参数设置为零不能创建jvm
在提交Spark任务脚本中没有指定Driver的内存所以在脚本中设置添加参数即可
网友评论