美文网首页
linux下安装kafka教程

linux下安装kafka教程

作者: 西葫芦炒胖子 | 来源:发表于2019-07-24 23:52 被阅读0次

github地址:https://github.com/douzixiansheng/MQ/blob/master/kafka_linux_install.md

Kafka linux 安装步骤

官方步骤:
http://kafka.apache.org/quickstart

安装步骤

  • 安装JDK
    Kafka 使用Zookeeper 来保存相关配置信息,Kafka及Zookeeper 依赖Java运行环境。
    <a>https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html</a>

    • 下载 JDK (jdk-8u221-linux-x64.tar.gz)

      解压
      tar -zxvf jdk-8u65-linux-x64.tar.gz
      mv jdk1.8.0_221 java
      
      配置Java环境变量
      vi /etc/profile
      
      在配置文件尾部增加
      export JAVA_HOME=/usr/local/software/java
      export PATH=$JAVA_HOME/bin:$PATH
      export CLASSPATH=:#JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
      
  • 下载Kafka

    wget http://labfile.oss.aliyuncs.com/courses/859/kafka_2.10-0.10.2.1.tgz
    
    • 解压Kafka
    tar -zxvf kafka_2.10-0.10.2.1.tgz 
    
    • 进入配置文件目录
    cd /usr/local/software/kafka_2.10-0.10.2.1/config
    
    • 修改配置文件server.properties
    log.dirs=/usr/local/logs/kafka
    zookeeper.connect=localhost:2181
    
  • 编写启动脚本kafka_start.sh

    #启动zookeeper
    /usr/local/software/kafka_2.10-0.10.2.1/bin/zookeeper-server-start.sh /usr/local/software/kafka_2.10-0.10.2.1/config/zookeeper.properties &
    #等3秒后执行
    sleep 3
    #启动kafka
    /usr/local/software/kafka_2.10-0.10.2.1/bin/kafka-server-start.sh /usr/local/software/kafka_2.10-0.10.2.1/config/server.properties &
    
  • 编写停止脚本kafka_stop.sh

    #关闭zookeeper
    /usr/local/software/kafka_2.10-0.10.2.1/bin/zookeeper-server-stop.sh /usr/local/software/kafka_2.10-0.10.2.1/config/zookeeper.properties &
    #等3秒后执行
    sleep 3
    #关闭kafka
    /usr/local/software/kafka_2.10-0.10.2.1/bin/kafka-server-stop.sh /usr/local/software/kafka_2.10-0.10.2.1/config/server.properties &
    
  • 启动kafka

    ./kafka_start.sh
    
  • 使用jps 查看

    root@FM:/usr/local/software# jps
    4649 Jps
    4377 Kafka
    4139 QuorumPeerMain
    

Kafka 使用

  • 创建主题
./kafka-topics.sh --create --topic mytopic2 --zookeeper localhost:2181 --partitions 3 --replication-factor 1
执行结果
Created topic "mytopic2".
  • 查看当前主题
 ./kafka-topics.sh --describe --zookeeper localhost:2181
 或只看主题名称
 ./kafka-topics.sh --list --zookeeper localhost:2181
执行结果
Topic:mytopic2  PartitionCount:3        ReplicationFactor:1     Configs:
    Topic: mytopic2 Partition: 0    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic2 Partition: 1    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic2 Partition: 2    Leader: 0       Replicas: 0     Isr: 0
Topic:mytopic3  PartitionCount:3        ReplicationFactor:1     Configs:
    Topic: mytopic3 Partition: 0    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic3 Partition: 1    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic3 Partition: 2    Leader: 0       Replicas: 0     Isr: 0
  • 查看某个主题是否存在

    ./kafka-topics.sh --list --zookeeper localhost:2181|grep mytopic11
    
  • 修改主题

    ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic2 --partitions 10
    
  • 删除配置参数

    ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic2 --delete-config flush.messages
    
  • 删除主题

    ./kafka-topics.sh --zookeeper localhost:2181 --delete --topic mytopic2
    结果:
    Topic mytopic2 is marked for deletion.
    Note: This will have no impact if delete.topic.enable is not set to true.
    
  • 启动kafka producer

     ./kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic3
    
  • 启动kafka consumer

    ./kafka-console-consumer.sh --zookeeper localhost --topic mytopic3
    
  • 终端写入kafka主题数据

    ./kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic3
    
  • 终端消费kafka主题数据

    ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic mytopic3 --from-beginning
    

相关文章

网友评论

      本文标题:linux下安装kafka教程

      本文链接:https://www.haomeiwen.com/subject/edvqrctx.html