美文网首页
linux下安装kafka教程

linux下安装kafka教程

作者: 西葫芦炒胖子 | 来源:发表于2019-07-24 23:52 被阅读0次

    github地址:https://github.com/douzixiansheng/MQ/blob/master/kafka_linux_install.md

    Kafka linux 安装步骤

    官方步骤:
    http://kafka.apache.org/quickstart

    安装步骤

    • 安装JDK
      Kafka 使用Zookeeper 来保存相关配置信息,Kafka及Zookeeper 依赖Java运行环境。
      <a>https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html</a>

      • 下载 JDK (jdk-8u221-linux-x64.tar.gz)

        解压
        tar -zxvf jdk-8u65-linux-x64.tar.gz
        mv jdk1.8.0_221 java
        
        配置Java环境变量
        vi /etc/profile
        
        在配置文件尾部增加
        export JAVA_HOME=/usr/local/software/java
        export PATH=$JAVA_HOME/bin:$PATH
        export CLASSPATH=:#JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
        
    • 下载Kafka

      wget http://labfile.oss.aliyuncs.com/courses/859/kafka_2.10-0.10.2.1.tgz
      
      • 解压Kafka
      tar -zxvf kafka_2.10-0.10.2.1.tgz 
      
      • 进入配置文件目录
      cd /usr/local/software/kafka_2.10-0.10.2.1/config
      
      • 修改配置文件server.properties
      log.dirs=/usr/local/logs/kafka
      zookeeper.connect=localhost:2181
      
    • 编写启动脚本kafka_start.sh

      #启动zookeeper
      /usr/local/software/kafka_2.10-0.10.2.1/bin/zookeeper-server-start.sh /usr/local/software/kafka_2.10-0.10.2.1/config/zookeeper.properties &
      #等3秒后执行
      sleep 3
      #启动kafka
      /usr/local/software/kafka_2.10-0.10.2.1/bin/kafka-server-start.sh /usr/local/software/kafka_2.10-0.10.2.1/config/server.properties &
      
    • 编写停止脚本kafka_stop.sh

      #关闭zookeeper
      /usr/local/software/kafka_2.10-0.10.2.1/bin/zookeeper-server-stop.sh /usr/local/software/kafka_2.10-0.10.2.1/config/zookeeper.properties &
      #等3秒后执行
      sleep 3
      #关闭kafka
      /usr/local/software/kafka_2.10-0.10.2.1/bin/kafka-server-stop.sh /usr/local/software/kafka_2.10-0.10.2.1/config/server.properties &
      
    • 启动kafka

      ./kafka_start.sh
      
    • 使用jps 查看

      root@FM:/usr/local/software# jps
      4649 Jps
      4377 Kafka
      4139 QuorumPeerMain
      

    Kafka 使用

    • 创建主题
    ./kafka-topics.sh --create --topic mytopic2 --zookeeper localhost:2181 --partitions 3 --replication-factor 1
    
    执行结果
    Created topic "mytopic2".
    
    • 查看当前主题
     ./kafka-topics.sh --describe --zookeeper localhost:2181
     或只看主题名称
     ./kafka-topics.sh --list --zookeeper localhost:2181
    
    执行结果
    Topic:mytopic2  PartitionCount:3        ReplicationFactor:1     Configs:
        Topic: mytopic2 Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: mytopic2 Partition: 1    Leader: 0       Replicas: 0     Isr: 0
        Topic: mytopic2 Partition: 2    Leader: 0       Replicas: 0     Isr: 0
    Topic:mytopic3  PartitionCount:3        ReplicationFactor:1     Configs:
        Topic: mytopic3 Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: mytopic3 Partition: 1    Leader: 0       Replicas: 0     Isr: 0
        Topic: mytopic3 Partition: 2    Leader: 0       Replicas: 0     Isr: 0
    
    • 查看某个主题是否存在

      ./kafka-topics.sh --list --zookeeper localhost:2181|grep mytopic11
      
    • 修改主题

      ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic2 --partitions 10
      
    • 删除配置参数

      ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic2 --delete-config flush.messages
      
    • 删除主题

      ./kafka-topics.sh --zookeeper localhost:2181 --delete --topic mytopic2
      结果:
      Topic mytopic2 is marked for deletion.
      Note: This will have no impact if delete.topic.enable is not set to true.
      
    • 启动kafka producer

       ./kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic3
      
    • 启动kafka consumer

      ./kafka-console-consumer.sh --zookeeper localhost --topic mytopic3
      
    • 终端写入kafka主题数据

      ./kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic3
      
    • 终端消费kafka主题数据

      ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic mytopic3 --from-beginning
      

    相关文章

      网友评论

          本文标题:linux下安装kafka教程

          本文链接:https://www.haomeiwen.com/subject/edvqrctx.html