本节主要内容:
用sqoop进行Mysql到HDFS的导入导出
一、准备数据
数据准备(Node3节点)
在mysql中建立sqoop_test库
# mysql -uroot -p123456
mysql> create database sqoop_test;
Query OK, 1 row affected (0.00 sec)
在sqoop_test里面建立一个表
mysql> use sqoop_test;
Database changed
mysql> CREATE TABLE `student_sqoop` (
-> `id` int(11) NOT NULL,
-> `name` varchar(20) NOT NULL,
-> PRIMARY KEY (`id`)
-> ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.04 sec)
插入数据
mysql> insert into student_sqoop (id,name) values (1,'michael');
mysql> insert into student_sqoop (id,name) values (2,'ted');
mysql> insert into student_sqoop (id,name) values (3,'jack');
二、导入mysql到hdfs
为了测试,mysql配置权限(Node3节点)
mysql> grant all privileges on *.* to root@'%' identified by '123456';
Query OK, 0 rows affected (0.00 sec)
导入(Node1节点)
# sudo -u hdfs sqoop import --connect jdbc:mysql://node3.hadoop.com/sqoop_test --username root --password 123456 --table student_sqoop --m 1 --target-dir /user/student_sqoop
确认导入
# sudo -u hdfs hadoop fs -ls /user/student_sqoop
Found 2 items
-rw-r--r-- 3 hdfs supergroup 0 2020-07-05 00:12 /user/student_sqoop/_SUCCESS
-rw-r--r-- 3 hdfs supergroup 23 2020-07-05 00:12 /user/student_sqoop/part-m-00000
# sudo -u hdfs hadoop fs -cat /user/student_sqoop/part-m-00000
1,michael
2,ted
3,jack
三、从hdfs导出到mysql
1.清空mysql相关表(Node3节点)
mysql> show tables;
+----------------------+
| Tables_in_sqoop_test |
+----------------------+
| student_sqoop |
+----------------------+
1 row in set (0.00 sec)
mysql> truncate student_sqoop;
Query OK, 0 rows affected (0.00 sec)
mysql> select * from student_sqoop;
Empty set (0.00 sec)
2.导出数据到mysql(Node1节点)
# sudo -u hdfs sqoop export --connect jdbc:mysql://node3.hadoop.com/sqoop_test --username root --password 123456 --table student_sqoop --m 1 --export-dir /user/student_sqoop
3.确认导出结果(Node3节点)
mysql> select * from student_sqoop;
+----+---------+
| id | name |
+----+---------+
| 1 | michael |
| 2 | ted |
| 3 | jack |
+----+---------+
3 rows in set (0.00 sec)
数据成功导入到mysql中
网友评论