kafka批量导出导入数据

批量导出数据

./bin/kafka-console-consumer.sh --bootstrap-server 10.5.3.204:9092,10.5.3.205:9092 --topic processJob_pkvspp_todo --consumer-property group.id=processjob_pkvspp_todo-null-consumer --from-beginning

批量导入数据

./kafka-console-producer.sh --broker-list 10.5.3.206:9092 --topic yutao < three_phone_pkvspp_result.kfk.txt

查看分区副本数

[root@swk-206 bin]# ./kafka-topics.sh --zookeeper 10.5.3.207:2181/kafka --topic qmswk-task --describe
Topic:qmswk-task    PartitionCount:4    ReplicationFactor:1 Configs:max.message.bytes=5242880
    Topic: qmswk-task   Partition: 0    Leader: 0   Replicas: 0 Isr: 0
    Topic: qmswk-task   Partition: 1    Leader: 0   Replicas: 0 Isr: 0
    Topic: qmswk-task   Partition: 2    Leader: 0   Replicas: 0 Isr: 0
    Topic: qmswk-task   Partition: 3    Leader: 0   Replicas: 0 Isr: 0
[root@swk-206 bin]# 

parition数量、replica因子以及每个partition的leader、replica信息

开发环境清理

./bin/kafka-console-consumer.sh --bootstrap-server 10.5.3.206:9092 --topic xk-vspp-tl-msvpp-in-qmswk --consumer-property group.id=xk-vspp-tl-msvpp-in-qmswk-qmswk_kfk_group-consumer

指定消费者组清理

./bin/kafka-console-consumer.sh --bootstrap-server 10.5.3.204:9092,10.5.3.205:9092 --topic xk-bz-tl-in-chin --consumer-property group.id=xk-bz-dec-tl-in-chin-tool-consumer --consumer-property auto.offset.reset=earliest >bz-dec-tl-in-chin-data.kfk

auto.offset.reset默认为latest