ubuntu 18.04 安装sqoop

前提

HADOOP 和MYSQL安装ok

下载

https://sqoop.apache.org

$ sudo cp sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz /opt/
$ sudo tar xzvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz 
$ sudo mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop
$ vi .bashrc 
    export SQOOP_HOME=/opt/sqoop
    export PATH=$PATH:$SQOOP_HOME/bin
    export SQOOP_CONF_DIR=$SQOOP_HOME/conf
    export SQOOP_CLASS_PATH=$SQOOP_CONF_DIR
$ source ~/.bashrc
$ cd $SQOOP_HOME/conf
$ mv sqoop-env-template.sh sqoop-env.sh
$ vi /opt/sqoop/conf/sqoop-env.sh
export HADOOP_COMMON_HOME=/opt/hadoop
export HADOOP_MAPRED_HOME=/opt/hadoop
$ tar -zxf mysql-connector-java-5.1.30.tar.gz
$ su
password:

# cd mysql-connector-java-8.0.16.jar
# mv mysql-connector-java-8.0.16.jar-bin.jar /opt/sqoop/lib

$ cd $SQOOP_HOME/bin
$ sqoop-version
...
2019-06-18 18:27:50,668 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

坑

• Sqoop 连接Mysql org.apache.commons.lang.StringUtils

下载 http://mirrors.tuna.tsinghua.edu.cn/apache//commons/lang/binaries/commons-lang-2.6-bin.zip

上传到sqoop的 lib目录。

• sqoop import says can find Class <tablename>

使用 --bindir ./ 可以搞定。比如

sqoop import --connect jdbc:mysql://localhost:3306/energydata --username hive --password 12345678_ --table average_price_by_state -m 1 --bindir .

参考资料

• python测试开发项目实战-目录
• python工具书籍下载-持续更新
• python 3.7极速入门教程 - 目录
• 讨论qq群630011153 144081101
• 原文地址
• 本文涉及的python测试开发库 谢谢点赞！
• 本文相关海量书籍下载
• https://www.tutorialspoint.com/sqoop/sqoop_installation.htm
• https://sqoop.apache.org/docs/1.4.7/SqoopUserGuide.html