美文网首页
hive执行sql:stage-1 map = 0 reduce

hive执行sql:stage-1 map = 0 reduce

作者: soycici | 来源:发表于2018-08-24 23:41 被阅读0次

问题描述:
在hive shell中执行sql,如
insert into person_base1 select * from person_base;

运行日志中一直显示如下信息

hive> insert into person_base1 select * from person_base;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = root_20180824151434_b0b979fe-7a85-4ac6-b924-33dd40897a28
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1535114415498_0379, Tracking URL = http://master:8088/proxy/application_1535114415498_0379/
Kill Command = /opt/hadoop-2.7.4/bin/hadoop job  -kill job_1535114415498_0379
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2018-08-24 15:14:57,985 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:15:58,823 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:16:59,579 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:18:00,262 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:19:00,912 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:20:01,566 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:21:02,214 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:22:02,841 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:23:03,480 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:24:04,120 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:25:04,711 Stage-1 map = 0%,  reduce = 0%
2018-08-24 15:26:05,313 Stage-1 map = 0%,  reduce = 0%

解决方式:
修改yarn-site.xml中的yarn.nodemanager.resource.memory-mb值大小(2G->4G)
原先设置

<property>
      <name>yarn.nodemanager.resource.memory-mb</name>
      <value>2048</value>
</property>

修改后设置

<property>
      <name>yarn.nodemanager.resource.memory-mb</name>
      <value>4096</value>
</property>

相关文章

  • hive执行sql:stage-1 map = 0 reduce

    问题描述:在hive shell中执行sql,如insert into person_base1 select *...

  • Hive简易教程 - 数据分析

    Hive是一个HDFS上的sql执行引擎,它将sql语句转化为Hadoop上的map-reduce任务来执行。由于...

  • Hive 任务卡在 map = 0%, reduce = 0%

    Hive 卡在map = 0%, reduce = 0%阶段 解决:增加map个数,设置mapreduce.in...

  • hive优化参数说明

    1,一个Hive查询生成多个Map Reduce Job,一个Map Reduce Job又有Map,Reduce...

  • Hive优化

    Hive HQL优化 Hive优化目标在有限的资源下,执行效率更高 常见问题数据倾斜map数设置reduce数设置...

  • hadoop文件系统HDFS

    Hadoop家族成员: HDFS Map/Reduce zookeeper Hive Hbase 这些...

  • hive 动态分区实现 (hive-1.1.0)

    笔者使用的hive版本是hive-1.1.0 hive-1.1.0动态分区的默认实现是只有map没有reduce,...

  • HADOOP-HIVE4

    Hive压缩 开启Map输出阶段压缩 开启map输出阶段压缩可以减少job中map和Reduce task间数据传...

  • hive调优

    hive的查询注意事项以及优化总结 . Hive是将符合SQL语法的字符串解析生成可以在Hadoop上执行的Map...

  • 配置hive-site.xml

    官网给出了Apache Hive的配置说明,Hive的配置支持多种方式,主要如下(以map-reduce临时目录配...

网友评论

      本文标题:hive执行sql:stage-1 map = 0 reduce

      本文链接:https://www.haomeiwen.com/subject/uyhwiftx.html