美文网首页
zookeeper删除kafka元数据

zookeeper删除kafka元数据

作者: 大数据技术进阶 | 来源:发表于2017-05-25 14:11 被阅读0次

    问题:卸载kafka前未删除kafka topic,重新安装kafka后,生成跟之前topic名字相同的topic时报错,显示topic已存在

    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
    WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
    Error while executing topic command : Topic "hive-test-table" already exists.
    [2017-01-16 10:48:30,766] ERROR kafka.common.TopicExistsException: Topic "hive-test-table" already exists.
        at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:261)
        at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:245)
        at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:107)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)
     (kafka.admin.TopicCommand$)
    
    

    原因:kafka topic的元数据存储在zookeeper里,卸载kafka前未删除topic,之前topic的元数据仍存在zk里,需要手动删除zk里的元数据

    解决:

    zookeeper shell命令

    [zk: d65:2181/kafka_hive(CONNECTED) 7] ll
    ZooKeeper -server host:port cmd args
        connect host:port
        get path [watch]
        ls path [watch]
        set path data [version]
        rmr path
        delquota [-n|-b] path
        quit 
        printwatches on|off
        create [-s] [-e] path data acl
        stat path [watch]
        close 
        ls2 path [watch]
        history 
        listquota path
        setAcl path acl
        getAcl path
        sync path
        redo cmdno
        addauth scheme auth
        delete path [version]
        setquota -n|-b val path
    

    进入zookeeper目录,删除kafka topic 元数据

    [root@d100 bin]# sh zkCli.sh -server  d65:2181/kafka_hive
    
    [zk: d65:2181/kafka_hive(CONNECTED) 8] ls /
    [consumers, config, controller, isr_change_notification, brokers, admin, controller_epoch]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 9] ls /brokers
    [seqid, topics, ids]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 10] ls /brokers/topics
    [略...]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 11] rmr /brokers/topics/hive-test-table
    
    [zk: d65:2181/kafka_hive(CONNECTED) 13] ls /brokers/topics
    [略...]
    

    重新创建topic成功

    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
    Created topic "hive-test-table".
    
    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --desc  --topic hive-test-table
    Topic:hive-test-table   PartitionCount:5    ReplicationFactor:2 Configs:
        Topic: hive-test-table  Partition: 0    Leader: 1285    Replicas: 1285,1281 Isr: 1285,1281
        Topic: hive-test-table  Partition: 1    Leader: 1281    Replicas: 1281,1282 Isr: 1281,1282
        Topic: hive-test-table  Partition: 2    Leader: 1282    Replicas: 1282,1283 Isr: 1282,1283
        Topic: hive-test-table  Partition: 3    Leader: 1283    Replicas: 1283,1284 Isr: 1283,1284
        Topic: hive-test-table  Partition: 4    Leader: 1284    Replicas: 1284,1285 Isr: 1284,1285
    

    相关文章

      网友评论

          本文标题:zookeeper删除kafka元数据

          本文链接:https://www.haomeiwen.com/subject/tnfzxxtx.html