12.Hadoop:sqoop部署

本节主要内容：

sqoop部署

sqoop是用于在传统关系型数据库跟hdfs之间进行数据导入导出的工具。

1.系统环境：

OS：CentOS Linux release 7.5.1804 (Core)

CPU：2核心

Memory：1GB

运行用户：root

JDK版本：1.8.0_252

Hadoop版本：cdh5.16.2

2.集群各节点角色规划为：

172.26.37.245 node1.hadoop.com---->namenode，zookeeper，journalnode，hadoop-hdfs-zkfc，resourcenode，historyserver，hbase，hbase-master，hive，hive-metastore，hive-server2,hive-hbase，sqoop

172.26.37.246 node2.hadoop.com---->datanode，zookeeper，journalnode，nodemanager，hadoop-client，mapreduce，hbase-regionserver

172.26.37.247  node3.hadoop.com---->datanode，nodemanager，hadoop-client，mapreduce，hive，mysql-server

172.26.37.248  node4.hadoop.com---->namenode，zookeeper，journalnode，hadoop-hdfs-zkfc，hive，hive-server2

3.环境说明：

本次追加部署

172.26.37.245 node1.hadoop.com---->sqoop

一.安装

node1节点

      # yum -y install sqoop

二.测试安装

      # sqoop version

Warning: /usr/lib/sqoop/../hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

19/04/23 23:38:08 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.2

Sqoop 1.4.6-cdh5.16.2

git commit id

Compiled by jenkins on Wed Jul 18 21:18:40 PST 2019

三.准备jdbc驱动

      # ln -s /usr/share/java/mysql-connector-java.jar /usr/lib/sqoop/lib/

      # ls /usr/lib/sqoop/lib/mysql-connector-java.jar

/usr/lib/sqoop/lib/mysql-connector-java.jar

默认缺少java-json.jar

      # wget http://www.java2s.com/Code/JarDownload/java-json/java-json.jar.zip

      # unzip java-json.jar.zip

下载解压后放到/usr/lib/sqoop/lib

      # cp java-json.jar /usr/lib/sqoop/lib/

变更权限

      # chmod 744 java-json.jar 改个权限