基于2.x hdfs HA的搭建:
1.配置mapred-site.xml,默认没有,只有一个 mapred-site.xml.template,mv mapred-site.xml.template mapred-site.xml,加入配置
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
2.配置yarn-site.xml:
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>cluster1</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>node08</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>node09</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>node07:2181,node08:2181,node09:2181</value>
</property>
3.启动,在hdfs已启动的情况下,在主节点执行start-yarn.sh 启动yarn,注意,此时只启动了nodeManager,resourceManager还需要在其配置的节点手动启动,yarn-daemon.sh start resourcemanager ,因为resourcemanager也是HA ,所以要在两个节点执行,这样就启动了。
测试,可在浏览器查看resourcemanager节点8088端口

网友评论