sqoop1.4.6安装

系统与环境

1. ubuntu 14.04
2. Apache Hadoop 2.7.3
3. Apache Zookeeper 3.4.9
4. Apache Hbase 1.2.4
5. Apache Hive 2.1.1
6. Apache sqoop 1.4.6

安装前期准备

jdk、hadoop、zookeeper、hbase、hive 均已经配置好

安装

1. 从官网下载安装包
   https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.4.6/
2. 根据hadoop的版本选择对应的sqoop
   根据hadoop下载对应版本
   因为已经安装了hadoop2.7.3 所以下载sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
3. 通过

wget https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

下载sqoop安装包

4. 将sqoop解压到安装目录

tar -xzvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /home/hadoop/deploy  

5. 在Sqoop根目录下，进入conf目录，找到其中的sqoop-env-template.sh，复制成sqoop-env.sh

cp sqoop-env-template.sh ./sqoop-env.sh

6. 修改sqoop-env.sh

#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=${HADOOP_HOME}
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=${HADOOP_HOME}
#set the path to where bin/hbase is available
export HBASE_HOME=${HBASE_HOME}
#Set the path to where bin/hive is available
export HIVE_HOME=${HIVE_HOME}
#Set the path for where zookeper config dir is
export ZOOCFGDIR=${ZOOKEEPER_HOME}/conf

7. 设置SQOOP_HOME这样方便可以直接使用sqoop命令
   通过sudo vim /etc/profile

export SQOOP_HOME=/home/hadoop/deploy/sqoop-1.4.6
export PATH=$PATH:$SQOOP_HOME/bin

8.通过source /etc/profile 使配置生效，然后通过sqoop help 可以验证是否安装成功。

异常

输入sqoop help 可能会报错：

Warning: /usr/bigData/sqoop1/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/bigData/sqoop1/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.

错误原因
集群没有安装HCatalog 和Accumulo

解决方案
修改sqoop安装目录bin文件夹下的configure-sqoop文件，找到如下位置并注释

## Moved to be a runtime check in sqoop.
#if [ ! -d "${HCAT_HOME}" ]; then
#  echo "Warning: $HCAT_HOME does not exist! HCatalog jobs will fail."
#  echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'
#fi

#if [ ! -d "${ACCUMULO_HOME}" ]; then
#  echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."
#  echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'
#fi