美文网首页
13.Hadoop:用sqoop进行Mysql到HDFS的导入导

13.Hadoop:用sqoop进行Mysql到HDFS的导入导

作者: 負笈在线 | 来源:发表于2020-07-06 07:20 被阅读0次

本节主要内容:

用sqoop进行Mysql到HDFS的导入导出

一、准备数据

数据准备(Node3节点)

在mysql中建立sqoop_test库

            # mysql -uroot -p123456

mysql> create database sqoop_test;

Query OK, 1 row affected (0.00 sec)

在sqoop_test里面建立一个表 

mysql> use sqoop_test;

Database changed

mysql> CREATE TABLE `student_sqoop` (   

    ->  `id` int(11) NOT NULL,   

    ->  `name` varchar(20) NOT NULL,   

    ->  PRIMARY KEY (`id`)   

    -> ) ENGINE=MyISAM  DEFAULT CHARSET=utf8; 

Query OK, 0 rows affected (0.04 sec)

插入数据

mysql> insert into student_sqoop (id,name) values (1,'michael'); 

mysql> insert into student_sqoop (id,name) values (2,'ted');

mysql> insert into student_sqoop (id,name) values (3,'jack');

二、导入mysql到hdfs

为了测试,mysql配置权限(Node3节点)

mysql> grant all privileges on *.* to root@'%' identified by '123456';

Query OK, 0 rows affected (0.00 sec)

导入(Node1节点)

        #  sudo -u hdfs sqoop import --connect jdbc:mysql://node3.hadoop.com/sqoop_test --username root --password 123456 --table student_sqoop --m 1 --target-dir /user/student_sqoop

确认导入

        #  sudo -u hdfs hadoop fs -ls /user/student_sqoop

Found 2 items

-rw-r--r--  3 hdfs supergroup          0 2020-07-05 00:12 /user/student_sqoop/_SUCCESS

-rw-r--r--  3 hdfs supergroup        23 2020-07-05 00:12 /user/student_sqoop/part-m-00000

        # sudo -u hdfs hadoop fs -cat /user/student_sqoop/part-m-00000

1,michael

2,ted

3,jack

三、从hdfs导出到mysql

1.清空mysql相关表(Node3节点)

mysql> show tables;

+----------------------+

| Tables_in_sqoop_test |

+----------------------+

| student_sqoop        |

+----------------------+

1 row in set (0.00 sec)

mysql> truncate student_sqoop;

Query OK, 0 rows affected (0.00 sec)

mysql> select * from student_sqoop;

Empty set (0.00 sec)

2.导出数据到mysql(Node1节点)

        # sudo -u hdfs sqoop export --connect jdbc:mysql://node3.hadoop.com/sqoop_test --username root --password 123456 --table student_sqoop --m 1 --export-dir /user/student_sqoop

3.确认导出结果(Node3节点)

mysql> select * from student_sqoop;

+----+---------+

| id | name    |

+----+---------+

|  1 | michael |

|  2 | ted    |

|  3 | jack    |

+----+---------+

3 rows in set (0.00 sec)

数据成功导入到mysql中

相关文章

网友评论

      本文标题:13.Hadoop:用sqoop进行Mysql到HDFS的导入导

      本文链接:https://www.haomeiwen.com/subject/lvtvqktx.html