Hive 安装

安装 Hadoop 集群

参考 Docker Hadoop 集群搭建

下载 Hive

Hive下载地址
本实例下载版本： apache-hive-3.1.1

root@suoron:~# tar zxvf apache-hive-3.1.1-bin.tar.gz -C /root/hive/apache-hive-3.1.1
#挂载到 Hadoop master 容器，脚本参考: https://www.jianshu.com/p/4bdd888fd81f
root@suoron:~# ./dynamic_mount_docker_volume.sh master /root/hive/apache-hive-3.1.1 /opt/apache-hive-3.1.1

安装 Mysql

参考: Linux 安装 MySQL
mysql 主要用来存储 Hive 元数据

• 新建数据库 hive
• 开通远程访问权限

export JAVA_HOME=/usr/local/jdk1.8.0_152
export HIVE_HOME=/opt/apache-hive-3.1.1
export HADOOP_HOME=/usr/local/hadoop-2.9.1
export PATH=$PATH:$HIVE_HOME/bin

• 复制 jdbc 驱动到 hive

root@47ba702846f0:/opt/apache-hive-3.1.1# ll lib/mysql-connector-java-5.1.46.jar 
-rw-r--r-- 1 root root 1004838 Jun 23 12:30 lib/mysql-connector-java-5.1.46.jar

配置环境变量

• 登录 master 容器

root@suoron:~#export JAVA_HOME=/usr/local/jdk1.8.0_152
root@suoron:~#export HIVE_HOME=/opt/apache-hive-3.1.1
root@suoron:~#export HADOOP_HOME=/usr/local/hadoop-2.9.1
root@suoron:~#export PATH=$PATH:$HIVE_HOME/bin

• 修改 hive-config.sh

root@47ba702846f0:/opt/apache-hive-3.1.1# vi bin/hive-config.sh
在文件末尾添加
export JAVA_HOME=/usr/local/jdk1.8.0_152
export HIVE_HOME=/opt/apache-hive-3.1.1
export HADOOP_HOME=/usr/local/hadoop-2.9.1
export PATH=$PATH:$HIVE_HOME/bin

• 修改 hive-site.xml

root@47ba702846f0:/opt/apache-hive-3.1.1/conf# cp hive-default.xml.template hive-site.xml
root@47ba702846f0:/opt/apache-hive-3.1.1/conf#vi hive-site.xml
# 在 </configuration> 前添加
  <property> 
    <name>javax.jdo.option.ConnectionURL</name>  
    <value>jdbc:mysql://192.168.0.1:3306/hive?characterEncoding=UTF-8</value> 
  </property>  
  <property> 
    <name>javax.jdo.option.ConnectionDriverName</name>  
    <value>com.mysql.jdbc.Driver</value> 
  </property>  
  <property> 
    <name>javax.jdo.option.ConnectionUserName</name>  
    <value>hive</value> 
  </property>  
  <property> 
    <name>javax.jdo.option.ConnectionPassword</name>  
    <value>123</value> 
  </property>  
  <property> 
    <name>datanucleus.schema.autoCreateAll</name>  
    <value>true</value> 
  </property>  
  <property> 
    <name>hive.metastore.schema.verification</name>  
    <value>false</value> 
  </property>  
  <property> 
    <name>hive.exec.local.scratchdir</name>  
    <value>/opt/apache-hive-3.1.1/tmp</value>  
    <description>Local scratch space for Hive jobs</description> 
  </property>  
  <property> 
    <name>hive.downloaded.resources.dir</name>  
    <value>/opt/apache-hive-3.1.1/tmp/resources</value>  
    <description>Temporary local directory for added resources in the remote file system.</description> 
  </property>  
  <property> 
    <name>hive.querylog.location</name>  
    <value>/opt/apache-hive-3.1.1/tmp</value>  
    <description>Location of Hive run time structured log file</description> 
  </property>  
  <property> 
    <name>hive.server2.logging.operation.log.location</name>  
    <value>/opt/apache-hive-3.1.1/tmp/operation_logs</value>  
    <description>Top level directory where operation logs are stored if logging functitonality is enabled</description> 
  </property> 

• 在hive目录中创建tmp临时目录
  root@47ba702846f0: mkdir /opt/apache-hive-3.1.1/tmp

初始化 Hive 元数据库

root@47ba702846f0: schematool -dbType mysql -initSchema

测试 Hive 是否安装成功

root@47ba702846f0: hive
hive> show databases;
OK
default
Time taken: 0.019 seconds, Fetched: 1 row(s)
hive> 

能出现上面的提示信息，说明 Hive 已经成功安装

Hive 远程访问

• 开启 Thrift 服务

root@47ba702846f0:# hive --service hiveserver2
# java 客户端可以访问 10000 端口，浏览器可以访问 10002 端口

• 启动hiveWebInterface，通过网页访问 hive

root@47ba702846f0:#hive --service hwi
# 127.0.0.1:9999/hwi