美文网首页
The coordinator is not available

The coordinator is not available

作者: 何德何能者 | 来源:发表于2020-05-08 14:55 被阅读0次

    kafka consumer group 无法消费数据
    打开debug日志查看具体原因(此处是logback配置)

    <logger name="com.alibaba.otter" level="DEBUG"/>
    <root level="DEBUG">
    ...
    </root>
    

    看到关键提示

    2020-05-08 10:11:15.596 DEBUG   --- [Thread-3] org.apache.kafka.clients.consumer.internals.AbstractCoordinator        [onSuccess] [741] : [Consumer clientId=consumer--4, groupId=group_async] Received FindCoordinator response ClientResponse(receivedTimeMs=1588903875595, latencyMs=4, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=3, clientId=consumer-group_async-4, correlationId=11), responseBody=FindCoordinatorResponseData(throttleTimeMs=0, errorCode=15, errorMessage='The coordinator is not available.', nodeId=-1, host='', port=-1))
    

    根据这个提示得知,是消费组协调leader不存在导致
    查看kafka __consumer_offset topic


    kafka_consume_offset.png

    发现确实有Leader:none的

    可能原因是 __consumer_offset topic的默认分区是50,但是备份只有1份; kafka集群部署了3台,也就是3个brokers
    消费者组连接kafka,并会请求某一台来查找Coordinator(协调者),如果连接的机器上没有备份就不会有Leader,就会出现
    找不到Coordinator(协调者)

    解决方式
    1、停止kafka, 修改kafka配置 config/server.properties添加

    修改成broker的数量

    offsets.topic.replication.factor=3

    2、删除zookeeper配置信息

    ./bin/zkCli.sh -server 127.0.0.1:2181

    删除/config/topics/__consumer_offsets

    delete /config/topics/__consumer_offsets

    删除/brokers/topics/__consumer_offsets

    deleteall /brokers/topics/__consumer_offsets

    3、重启kafka

    参考
    https://stackoverflow.com/questions/51952398/kafka-connect-distributed-mode-the-group-coordinator-is-not-available
    https://www.cnblogs.com/jun1019/p/6634545.html
    https://blog.csdn.net/yf_li123/article/details/101552437
    https://www.orchome.com/1296
    https://github.com/edenhill/librdkafka/issues/2086

    相关文章

      网友评论

          本文标题:The coordinator is not available

          本文链接:https://www.haomeiwen.com/subject/goyfnhtx.html