美文网首页工作生活
Hadoop---yarn单机安装文档

Hadoop---yarn单机安装文档

作者: 吃货大米饭 | 来源:发表于2019-07-04 17:20 被阅读0次

本文紧接上文Hadoop---hdfs单机安装文档

1.修改mapred-site.xml文件

[hadoop@Hadoop001 hadoop]$ cp mapred-site.xml.template mapred-site.xml
[hadoop@Hadoop001 hadoop]$ vi mapred-site.xml
<configuration>
   <property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
   </property>
</configuration>

2.修改yarn-site.xml文件

[hadoop@Hadoop001 hadoop]$ vi yarn-site.xml 
<configuration>
   <property>
       <name>yarn.nodemanager.aux-services</name>
       <value>mapreduce_shuffle</value>
   </property>
</configuration>

3.开启ResourceManager和NodeManager进程

[hadoop@Hadoop001 hadoop]$ sbin/start-yarn.sh 

4.浏览ResourceManager的web界面,默认为8088端口

2019-07-04_170301.png

5.运行一个mr任务

[hadoop@Hadoop001 hadoop]$ find ./ -name '*example*.jar'
./share/hadoop/mapreduce1/hadoop-examples-2.6.0-mr1-cdh5.7.0.jar
./share/hadoop/mapreduce2/sources/hadoop-mapreduce-examples-2.6.0-cdh5.7.0-test-sources.jar
./share/hadoop/mapreduce2/sources/hadoop-mapreduce-examples-2.6.0-cdh5.7.0-sources.jar
./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -mkdir -p /examples/input
19/07/04 17:06:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -mkdir -p /examples/output
19/07/04 17:07:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -put wordcount.txt /examples/input/
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -cat /examples/input/wordcount.txt
19/07/04 17:18:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hello alvin
alvin fure
[hadoop@Hadoop001 hadoop]$ bin/hadoop jar ./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar wordcount /examples/input/* /examples/output/1
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -cat /examples/output/1/*
19/07/04 17:19:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
alvin   2
fure    1
hello   1

6.关闭yarn

[hadoop@Hadoop001 hadoop]$ sbin/stop-yarn.sh 
stopping yarn daemons
stopping resourcemanager
Hadoop001: stopping nodemanager
Hadoop001: nodemanager did not stop gracefully after 5 seconds: killing with kill -9
no proxyserver to stop

相关文章

网友评论

    本文标题:Hadoop---yarn单机安装文档

    本文链接:https://www.haomeiwen.com/subject/nxvrhctx.html