本文紧接上文Hadoop---hdfs单机安装文档
1.修改mapred-site.xml文件
[hadoop@Hadoop001 hadoop]$ cp mapred-site.xml.template mapred-site.xml
[hadoop@Hadoop001 hadoop]$ vi mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
2.修改yarn-site.xml文件
[hadoop@Hadoop001 hadoop]$ vi yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
3.开启ResourceManager和NodeManager进程
[hadoop@Hadoop001 hadoop]$ sbin/start-yarn.sh
4.浏览ResourceManager的web界面,默认为8088端口
![](https://img.haomeiwen.com/i11706106/5ad26367be112941.png)
5.运行一个mr任务
[hadoop@Hadoop001 hadoop]$ find ./ -name '*example*.jar'
./share/hadoop/mapreduce1/hadoop-examples-2.6.0-mr1-cdh5.7.0.jar
./share/hadoop/mapreduce2/sources/hadoop-mapreduce-examples-2.6.0-cdh5.7.0-test-sources.jar
./share/hadoop/mapreduce2/sources/hadoop-mapreduce-examples-2.6.0-cdh5.7.0-sources.jar
./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -mkdir -p /examples/input
19/07/04 17:06:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -mkdir -p /examples/output
19/07/04 17:07:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -put wordcount.txt /examples/input/
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -cat /examples/input/wordcount.txt
19/07/04 17:18:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hello alvin
alvin fure
[hadoop@Hadoop001 hadoop]$ bin/hadoop jar ./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar wordcount /examples/input/* /examples/output/1
[hadoop@Hadoop001 hadoop]$ bin/hdfs dfs -cat /examples/output/1/*
19/07/04 17:19:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
alvin 2
fure 1
hello 1
6.关闭yarn
[hadoop@Hadoop001 hadoop]$ sbin/stop-yarn.sh
stopping yarn daemons
stopping resourcemanager
Hadoop001: stopping nodemanager
Hadoop001: nodemanager did not stop gracefully after 5 seconds: killing with kill -9
no proxyserver to stop
网友评论