Kafka命令行操作

topic

创建topic

--topic topic名 --partitions 分区数 --replication-factor 副本数

[server@hadoop102 ~]$ kafka-topics.sh --create --zookeeper hadoop102:2181 --topic travel --partitions 2 --replication-factor 2
Created topic "travel".

查看当前服务器中的所有topic

[server@hadoop102 ~]$ kafka-topics.sh --list --zookeeper hadoop102:2181                                             travel

/opt/module/kafka/data
/opt/module/kafka/data
/opt/module/kafka/data

删除topic

[server@hadoop102 data]$ kafka-topics.sh --delete --zookeeper hadoop102:2181 --topic travel
Topic travel is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

再次创建新的

[server@hadoop102 data]$ kafka-topics.sh --create --zookeeper hadoop102:2181 --topic travel --partitions 3 --replication-factor 1
Created topic "travel".

/opt/module/kafka/data
/opt/module/kafka/data
/opt/module/kafka/data

查看某个topic的详情

[server@hadoop102 data]$ kafka-topics.sh --describe --topic travel -zookeeper hadoop102:2181         
Topic:travel    PartitionCount:3        ReplicationFactor:1     Configs:
        Topic: travel   Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: travel   Partition: 1    Leader: 1       Replicas: 1     Isr: 1
        Topic: travel   Partition: 2    Leader: 2       Replicas: 2     Isr: 2

[server@hadoop102 data]$ kafka-topics.sh --create --zookeeper hadoop102:2181 --topic eat --partitions 4 --replication-factor 3
Created topic "eat".
[server@hadoop102 data]$ kafka-topics.sh --describe --topic eat -zookeeper hadoop102:2181                           Topic:eat       PartitionCount:4        ReplicationFactor:3     Configs:
        Topic: eat      Partition: 0    Leader: 0       Replicas: 0,1,2 Isr: 0,1,2
        Topic: eat      Partition: 1    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: eat      Partition: 2    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: eat      Partition: 3    Leader: 0       Replicas: 0,2,1 Isr: 0,2,1

修改分区数,注意不可改小，否则报错The number of partitions for a topic can only be increased

[server@hadoop102 data]$ kafka-topics.sh --zookeeper hadoop102:2181 --alter --topic eat --partitions 6
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!
[server@hadoop102 data]$ kafka-topics.sh --describe --topic eat -zookeeper hadoop102:2181
Topic:eat       PartitionCount:6        ReplicationFactor:3     Configs:
        Topic: eat      Partition: 0    Leader: 0       Replicas: 0,1,2 Isr: 0,1,2
        Topic: eat      Partition: 1    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: eat      Partition: 2    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: eat      Partition: 3    Leader: 0       Replicas: 0,2,1 Isr: 0,2,1
        Topic: eat      Partition: 4    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: eat      Partition: 5    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1

生产者消费者测试

生产者

[server@hadoop102 data]$ kafka-console-producer.sh --topic eat --broker-list hadoop102:9092
>apple
>banana

消费者

[server@hadoop103 data]$ kafka-console-consumer.sh --topic eat --bootstrap-server hadoop102:9092

生产者

[server@hadoop102 data]$ kafka-console-producer.sh --topic eat --broker-list hadoop102:9092
>apple
>banana
>orange

消费者

[server@hadoop103 data]$ kafka-console-consumer.sh --topic eat --bootstrap-server hadoop102:9092
orange

消费者-从最开始开始消费

[server@hadoop104 data]$ kafka-console-consumer.sh --topic eat --bootstrap-server hadoop102:9092 --from-beginning
banana
orange
apple