美文网首页
sqoop 导入数据到 hive, 报错 Container k

sqoop 导入数据到 hive, 报错 Container k

作者: cooooper | 来源:发表于2019-04-15 11:19 被阅读0次

    sqoop 导入数据到 hive,报错 Container killed on request. Exit code is 143

    报错信息:

    19/04/10 09:49:01 INFO mapreduce.Job:  map 0% reduce 0%
    19/04/10 09:55:06 INFO mapreduce.Job: Task Id : attempt_1550821054018_1227_m_000000_0, Status : FAILED
    [2019-04-10 09:55:04.761]Container [pid=56901,containerID=container_1550821054018_1227_01_000002] is running 8163328B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used. Killing container.
    Dump of the process-tree for container_1550821054018_1227_01_000002 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 56901 56893 56901 56901 (bash) 3 9 9789440 293 /bin/bash -c /usr/java/jdk1.8.0_141-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN  -Djava.net.preferIPv4Stack=true -Xmx820m -Djava.io.tmpdir=/yarn/nm/usercache/impala/appcache/application_1550821054018_1227/container_1550821054018_1227_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 10.252.96.54 36388 attempt_1550821054018_1227_m_000000_0 2 1>/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002/stdout 2>/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002/stderr  
        |- 59095 56901 56901 56901 (java) 16567 714 2741301248 263844 /usr/java/jdk1.8.0_141-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Djava.net.preferIPv4Stack=true -Xmx820m -Djava.io.tmpdir=/yarn/nm/usercache/impala/appcache/application_1550821054018_1227/container_1550821054018_1227_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/yarn/container-logs/application_1550821054018_1227/container_1550821054018_1227_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 10.252.96.54 36388 attempt_1550821054018_1227_m_000000_0 2 
    
    [2019-04-10 09:55:04.774]Container killed on request. Exit code is 143
    [2019-04-10 09:55:04.785]Container exited with a non-zero exit code 143. 
    

    原因:
    内存分配不足,需要修改内存配置

    解决方法:
    修改CDH Yarn配置,将Map和Reduce任务的内存调大。



    然后重启过期服务即可。

    相关文章

      网友评论

          本文标题:sqoop 导入数据到 hive, 报错 Container k

          本文链接:https://www.haomeiwen.com/subject/wtzgiqtx.html