kafka安装配置

一、解压kafka包与配置home环境

二、server.properties  
1)broker.id=102
2) log.dirs=/opt/module/kafka/logs 
3)zookeeper.connect=hadoop102:2181,hadoop103:2181,hadoop104:2181

三、启动 
1)kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties 
2)kafka-server-stop.sh 

四、群起脚本 
for i in `cat /opt/module/hadoop-2.9.2/etc/hadoop/slaves` 
do 
echo "=============$i============" 
ssh $i `/opt/module/kafka/bin/kafka-server-start.sh -daemon /opt/module/kafka/config/server.properties` 
echo $?
done

五、命令操作 
1)bin/kafka-topics.sh --bootstrap-server hadoop102:2181 --list 
2)bin/kafka-topics.sh --bootstrap-server hadoop102:2181 --create --topic demo --partitions 2 --replication-factor 2 
3) bin/kafka-topics.sh --bootstrap-server hadoop102:2181 --describe --topic demo 
4)生成者: kafka-console-producer.sh --bootstrap-server hadoop102:9092 --topic bigdata 
5)消费者:kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic bigdata

六、依赖引入  
<groupId>org.apache.kafka</groupId>

<artifactId>kafka-clients</artifactId>

<version>2.6.0</version>

<groupId>com.fasterxml.jackson.core</groupId>

<artifactId>jackson-databind</artifactId>

<version>2.7.8</version>