美文网首页kafka
Kafka动态调整topic分区partition

Kafka动态调整topic分区partition

作者: 不1见2不3散4 | 来源:发表于2018-10-13 14:49 被阅读0次

    版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/russle/article/details/83038506

    我们在使用kafka时,初期创建时所指定topic属性需要修改,如何动态修改kafka属性?kafka提供了命令行工具—kafka-topics.sh.

    kafka-topics.sh工具介绍
    

    kafka-topics.sh工具也是我们用来创建topic、查看topic详情的工具。
    直接运行kafka-topics.sh可以产出它是用来创建、删除、查看以及更新topic
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh
    Create, delete, describe, or change a topic.
    Option Description

    –alter Alter the number of partitions,
    replica assignment, and/or
    configuration for the topic.
    –config <String: name=value> A topic configuration override for

    更新或者修改topic
    注意:我的kafka版本是1.1.0, 并且我只有一个broker。
    1, 首先我们创建一个topic,然后查看详情
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --create --zookeeper 192.168.119.131:2181 --replication-factor 1 --partitions 4 --topic yqtopic1
    Created topic “yqtopic1”.
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin#
    

    root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --describe --zookeeper 192.168.119.131:2181 --topic yqtopic1
    Topic:yqtopic1 PartitionCount:4 ReplicationFactor:1 Configs:
    Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

    kafka分区数据-before.png

    2,修改刚创建的topic,并查看修改的情况
    将分区数有4修改为12
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --alter --zookeeper 192.168.119.131:2181 --topic yqtopic1 --partitions 12
    WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
    Adding partitions succeeded!
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

    root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --describe --zookeeper 192.168.119.131:2181 --topic yqtopic1 Topic:yqtopic1 PartitionCount:12 ReplicationFactor:1 Configs:
    Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 4 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 5 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 6 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 7 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 8 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 9 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 10 Leader: 0 Replicas: 0 Isr: 0
    Topic: yqtopic1 Partition: 11 Leader: 0 Replicas: 0 Isr: 0
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ls -al /tmp/kafka-logs/
    total 72
    drwxr-xr-x 14 root root 4096 Oct 13 14:34 .
    drwxrwxrwt 17 root root 4096 Oct 13 14:34 …
    -rw-r–r-- 1 root root 0 Oct 13 14:10 cleaner-offset-checkpoint
    -rw-r–r-- 1 root root 0 Oct 13 14:10 .lock
    -rw-r–r-- 1 root root 4 Oct 13 14:33 log-start-offset-checkpoint
    -rw-r–r-- 1 root root 54 Oct 13 14:10 meta.properties
    -rw-r–r-- 1 root root 163 Oct 13 14:33 recovery-point-offset-checkpoint
    -rw-r–r-- 1 root root 163 Oct 13 14:34 replication-offset-checkpoint
    drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-0
    drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-1
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-10
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-11
    drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-2
    drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-3
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-4
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-5
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-6
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-7
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-8
    drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-9
    root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

    修改后的截图如下


    alter-partition.png

    作者:russle
    来源:CSDN
    原文:https://blog.csdn.net/russle/article/details/83038506?utm_source=copy
    版权声明:本文为博主原创文章,转载请附上博文链接!

    相关文章

      网友评论

        本文标题:Kafka动态调整topic分区partition

        本文链接:https://www.haomeiwen.com/subject/biuoaftx.html