Hadoop本身并没有提供HA的功能，需要借助ZooKeeper来实现Hadoop的HA功能。Hadoop的HA搭建过程是所有Hadoop生态圈组件中最复杂的，本节就来详细说明如何使用ZooKeeper来搭建Hadoop的HA环境。
环境说明：
bigdata131 192.168.126.131
bigdata132 192.168.126.132
bigdata133 192.168.126.133
bigdata134 192.168.126.134
安装介质下载：
zookeeper-3.4.10.tar.gz 提取码：nvv2
hadoop-2.7.3.tar.gz 提取码：r8xo
1.使用ZooKeeper实现Hadoop的HA的原理
image
通过上图的分析，使用ZooKeeper集群搭建一个最小规模的Hadoop HA集群至少需要4台机器：
Zookeeper集群：
bigdata131
bigdata132
bigdata133
Hadoop集群：
bigdata131 NameNode1 ResourceManager1 Journalnode
bigdata132 NameNode2 ResourceManager2 Journalnode
bigdata133 DataNode1
bigdata134 DataNode2
2.搭建ZooKeeper集群
参考文章《ZooKeeper从入门到精通9：ZooKeeper环境搭建之集群模式》。
3.搭建Hadoop的HA集群环境
以下步骤在bigdata131节点上执行：
3.1上传Hadoop安装包
使用winscp工具将Hadoop安装包上传到bigdata131节点的/root/tools/目录中，该目录是事先创建的。
# ls /root/tools/
hadoop-2.7.3.tar.gz
3.2解压Hadoop安装包
进入/root/tools/目录，将hadoop安装包解压到/root/trainings/目录中，该目录也是事先创建的。
# cd /root/tools/
# tar -zxvf hadoop-2.7.3.tar.gz -C /root/trainings/
3.3配置Hadoop环境变量（4台主机上都做一遍）
# cd /root/trainings/hadoop-2.7.3/
# pwd
/root/trainings/hadoop-2.7.3
# vim /root/.bash_profile
在文件末尾追加如下内容：
HADOOP_HOME=/root/trainings/hadoop-2.7.3
export HADOOP_HOME
PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
export PATH
按Esc:wq保存退出，使用source命令使配置文件立即生效：
# source /root/.bash_profile
3.4配置Hadoop HA模式的参数
进入Hadoop配置文件目录：
# cd /root/trainings/hadoop-2.7.3/etc/hadoop/
(1)配置hadoop-env.sh文件：
# echo $JAVA_HOME
/root/trainings/jdk1.8.0_144
# vim hadoop-env.sh
#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/root/trainings/jdk1.8.0_144
(2)配置hdfs-site.xml文件（配置nameservice中有几个namenode）：
# vim hdfs-site.xml
<configuration>
<property>
<name>dfs.nameservices</name>
<value>ns1</value>
</property>
<property>
<name>dfs.ha.namenodes.ns1</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn1</name>
<value>bigdata131:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn1</name>
<value>bigdata131:50070</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn2</name>
<value>bigdata132:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn2</name>
<value>bigdata132:50070</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://bigdata131:8485;bigdata132:8485;/ns1</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/root/trainings/hadoop-2.7.3/journal</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.ns1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>
sshfence
shell(/bin/true)
</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>30000</value>
</property>
</configuration>
(3)配置core-site.xml文件：
# mkdir /root/trainings/hadoop-2.7.3/tmp
# vim core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://ns1</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/root/trainings/hadoop-2.7.3/tmp</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>bigdata131:2181,bigdata132:2181,bigdata133:2181</value>
</property>
</configuration>
(4)配置mapred-site.xml文件：
将模板文件mapred-site.xml.template拷贝一份重命名为mapred-site.xml然后编辑：
# cp mapred-site.xml.template mapred-site.xml
# vim mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
(5)配置yarn-site.xml文件：
# vim yarn-site.xml
<configuration>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yrc</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>bigdata131</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>bigdata132</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>bigdata131:2181,bigdata132:2181,bigdata133:2181</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
(6)配置slaves文件：
# vim slaves
bigdata133
bigdata134
3.5将配置好的hadoop拷贝到其他节点
[root@bigdata131 ~]# scp -r /root/trainings/hadoop-2.7.3/ root@bigdata132:/root/trainings/
[root@bigdata131 ~]# scp -r /root/trainings/hadoop-2.7.3/ root@bigdata133:/root/trainings/
[root@bigdata131 ~]# scp -r /root/trainings/hadoop-2.7.3/ root@bigdata134:/root/trainings/
3.6启动Zookeeper集群
[root@bigdata131 ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/trainings/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@bigdata132 ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/trainings/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@bigdata133 ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/trainings/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
3.7在bigdata131和bigdata132上启动journalnode
[root@bigdata131 ~]# hadoop-daemon.sh start journalnode
starting journalnode, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-journalnode-bigdata131.out
[root@bigdata132 ~]# hadoop-daemon.sh start journalnode
starting journalnode, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-journalnode-bigdata132.out
3.8格式化HDFS和ZooKeeper（在bigdata131上执行）
[root@bigdata131 ~]# hdfs namenode -format
18/12/02 00:08:47 INFO common.Storage: Storage directory /root/trainings/hadoop-2.7.3/tmp/dfs/name has been successfully formatted.
[root@bigdata131 ~]# scp -r /root/trainings/hadoop-2.7.3/tmp root@bigdata132:/root/trainings/hadoop-2.7.3/
# 格式化zookeeper
[root@bigdata131 ~]# hdfs zkfc -formatZK
18/12/02 00:09:59 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/ns1 in ZK.
3.9启动Hadoop HA集群
在bigdata131上启动Hadoop集群：
[root@bigdata131 ~]# start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [bigdata131 bigdata132]
bigdata131: starting namenode, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-namenode-bigdata131.out
bigdata132: starting namenode, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-namenode-bigdata132.out
bigdata133: starting datanode, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-datanode-bigdata133.out
bigdata134: starting datanode, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-datanode-bigdata134.out
Starting journal nodes [bigdata131 bigdata132 ]
bigdata131: journalnode running as process 1275. Stop it first.
bigdata132: journalnode running as process 1265. Stop it first.
Starting ZK Failover Controllers on NN hosts [bigdata131 bigdata132]
bigdata131: starting zkfc, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-zkfc-bigdata131.out
bigdata132: starting zkfc, logging to /root/trainings/hadoop-2.7.3/logs/hadoop-root-zkfc-bigdata132.out
starting yarn daemons
starting resourcemanager, logging to /root/trainings/hadoop-2.7.3/logs/yarn-root-resourcemanager-bigdata131.out
bigdata133: starting nodemanager, logging to /root/trainings/hadoop-2.7.3/logs/yarn-root-nodemanager-bigdata133.out
bigdata134: starting nodemanager, logging to /root/trainings/hadoop-2.7.3/logs/yarn-root-nodemanager-bigdata134.out
[root@bigdata131 ~]# jps
1232 QuorumPeerMain
1939 ResourceManager
1524 NameNode
2200 Jps
1275 JournalNode
1839 DFSZKFailoverController
[root@bigdata132 ~]# jps
1217 QuorumPeerMain
1265 JournalNode
1346 NameNode
1461 DFSZKFailoverController
1518 Jps
[root@bigdata133 ~]# jps
1365 NodeManager
1213 QuorumPeerMain
1469 Jps
1263 DataNode
[root@bigdata134 ~]# jps
1303 NodeManager
1435 Jps
1228 DataNode
在bigdata132上需要单独启动ResourceManager：
[root@bigdata132 ~]# yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /root/trainings/hadoop-2.7.3/logs/yarn-root-resourcemanager-bigdata132.out
[root@bigdata132 ~]# jps
1217 QuorumPeerMain
1265 JournalNode
1346 NameNode
1603 Jps
1556 ResourceManager
1461 DFSZKFailoverController
4.测试Hadoop的HA集群环境
（1）正常情况下
image
image
image
可以看到：ZooKeeper中ns1当前Active的是bigdata131；通过hadoop网页发现：bigdata131是active状态，bigdata132是standby状态。
（2）杀死bigdata131上的NameNode进程，刷新ZooKeeper和网页观察变化
[root@bigdata131 ~]# jps
1232 QuorumPeerMain
1939 ResourceManager
1524 NameNode
2266 Jps
1275 JournalNode
1839 DFSZKFailoverController
[root@bigdata131 ~]# kill -9 1524
image
image
image
可以看到：ZooKeeper中ns1当前Active的变成了bigdata132；通过hadoop网页发现：bigdata131已经无法访问，bigdata132变成active状态。
因此Hadoop HA能够实现正确的失败迁移功能，可以更加高可用的对外提供Hadoop服务了。
至此，使用ZooKeeper搭建Hadoop的HA环境已经介绍完毕。祝你玩得愉快！
网友评论