配置zookeeper，启动多个HiveServer
配置HA Zookeeper信息如下：
<property>
<name>hive.server2.transport.mode</name>
<value>binary</value>
</property>
<property>
<name>hive.server2.zookeeper.namespace</name>
<value>hiveserver2-lsm</value>
</property>
<property>
<name>hive.vectorized.execution.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.zookeeper.quorum</name>
<value>hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181</value>
</property>
首先，在host1上启动MetaStore和HiveServer进程；
然后，在另外一台机器上启动HiveServer进程，组成简单的HA集群；
2017-06-27T10:31:18,263 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/home/hzlishuming/env/apache-hive-2.1.1-bin
2017-06-27T10:31:18,264 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181 sessionTimeout=1200000 watcher=org.apache.curator.ConnectionState@3f9b7fe1
2017-06-27T10:31:18,264 INFO [Thread-11] zookeeper.ZooKeeper: Initiating client connection, connectString=hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@47b17830
2017-06-27T10:31:18,295 INFO [main-SendThread(hadoop711.lt.163.org:2181)] zookeeper.Login: successfully logged in.
2017-06-27T10:31:18,296 INFO [Thread-12] zookeeper.Login: TGT refresh thread started.
2017-06-27T10:31:18,297 INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] zookeeper.Login: successfully logged in.
2017-06-27T10:31:18,298 INFO [Thread-13] zookeeper.Login: TGT refresh thread started.
2017-06-27T10:31:18,301 INFO [main-SendThread(hadoop711.lt.163.org:2181)] client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
2017-06-27T10:31:18,301 INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
2017-06-27T10:31:18,304 INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] zookeeper.ClientCnxn: Opening socket connection to server hadoop710.lt.163.org/10.120.219.54:2181. Will attempt to SASL-authenticate using Login Context section 'HiveZooKeeperClient'
2017-06-27T10:31:18,304 INFO [main-SendThread(hadoop711.lt.163.org:2181)] zookeeper.ClientCnxn: Opening socket connection to server hadoop711.lt.163.org/10.120.219.55:2181. Will attempt to SASL-authenticate using Login Context section 'HiveZooKeeperClient'
2017-06-27T10:31:18,304 INFO [Thread-11-SendThread(hadoop710.lt.163.org:2181)] zookeeper.ClientCnxn: Socket connection established to hadoop710.lt.163.org/10.120.219.54:2181, initiating session
2017-06-27T10:31:18,304 INFO [main-SendThread(hadoop711.lt.163.org:2181)] zookeeper.ClientCnxn: Socket connection established to hadoop711.lt.163.org/10.120.219.55:2181, initiating session
Beeline连接
./bin/beeline !connect jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm hadoop ""
beeline> !connect jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm hadoop ""
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hzlishuming/env/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/ndp/0.1.0/yarn_client/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm
17/06/27 14:04:09 [main]: INFO jdbc.HiveConnection: Connected to hadoop692.lt.163.org:10000
Connected to: Apache Hive (version 2.1.1)
Driver: Hive JDBC (version 2.1.1)
17/06/27 14:04:09 [main]: WARN jdbc.HiveConnection: Request to set autoCommit to false; Hive does not support autoCommit=false.
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop710.lt.163.org:2181,had> show databases;
+--------------------+--+
| database_name |
+--------------------+--+
| default |
| hive_examples |
| tpcds_data_test_2 |
+--------------------+--+
3 rows selected (2.017 seconds)
Zookeeper状态
目前有两台机器组成的集群，查看Zookeeper中的状态信息：
[zk: hadoop712.lt.163.org(CONNECTED) 1] ls /hiveserver2-lsm
[serverUri=hadoop691.lt.163.org:10000;version=2.1.1;sequence=0000000003, serverUri=hadoop692.lt.163.org:10000;version=2.1.1;sequence=0000000002]
[zk: hadoop712.lt.163.org(CONNECTED) 2] get /hiveserver2-lsm/serverUri=hadoop691.lt.163.org:10000;version=2.1.1;sequence=0000000003
hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hadoop691.lt.163.org;hive.server2.thrift.port=10000;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_HOST@TEST.AMBARI.NETEASE.COM
cZxid = 0x70001d53d
ctime = Tue Jun 27 11:02:34 CST 2017
mZxid = 0x70001d53d
mtime = Tue Jun 27 11:02:34 CST 2017
pZxid = 0x70001d53d
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x25cc3695ce40f40
dataLength = 296
numChildren = 0
高可用性模拟
当前Session正连接与
hadoop692.lt.163.org机器，现将该机器上的hiveserver进程kill掉，观察当前Session以及后续创建的Session会有如何影响？
当前Session处理会出现异常，如下：
0: jdbc:hive2://hadoop710.lt.163.org:2181,had> show databases;Unexpected end of file when reading from HS2 server. The root cause might be too many concurrent connections. Please ask the administrator to check the number of active connections, and adjust hive.server2.thrift.max.worker.threads if applicable.
Error: org.apache.thrift.transport.TTransportException (state=08S01,code=0)
再次创建Session，连接另外一台Host成功，符合预期：
hzlishuming@hadoop691:~/env/hive$ ./bin/beeline
Beeline version 2.1.1 by Apache Hive
beeline> !connect jdbc:hive2://hadoop710.lt.163.org:2181,hadoop711.lt.163.org:2181,hadoop712.lt.163.org:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-lsm hadoop ""
17/06/28 10:06:30 [main]: INFO jdbc.HiveConnection: Connected to hadoop691.lt.163.org:10000
Connected to: Apache Hive (version 2.1.1)
Driver: Hive JDBC (version 2.1.1)
17/06/28 10:06:30 [main]: WARN jdbc.HiveConnection: Request to set autoCommit to false; Hive does not support autoCommit=false.
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop710.lt.163.org:2181,had> show databases;
总结
- HiveServer2 HA模型配置、实现较为简单，是基于负载均衡的方式实现路由高可用的，Zookeeper层保存各节点的Host+Port信息，在创建Session的时候随机从中选择一台Host进行连接；
- 当有机器挂掉时，会在Zookeeper中删除该节点，实现高可用性能；
参考：
网友评论