需要jdk环境,kafka是用java和Scala编写的。
linux安装kafka
tar -zxvf kafka_2.12-1.0.2.tgz (或别的版本)
移动到opt/kafka
kafka环境变量 /etc/profile
KAFKA_HOME=/opt/kafka
PATH=$PATH:$KAFKA_HOME/bin
修改zookeeper文件位置,并创建相应文件夹
dataDir=/opt/zookeeper/data
修改kafka日志位置
log.dirs=/opt/kafka/kafka-logs
启动zookeeper kafka下 (-daemon后台启动)
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
INFO Got user-level KeeperException when processing sessionid:0x1805e7a3b4b0000 type:create cxid:0x49 zxid:0xc8 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
这不是一个报错故障,只是一个user-level KeeperException。可以忽略不做处理的
kafka安装好后,第一次启动。zookeeper日志Error:KeeperErrorCode = NoNode for /config/topics/test,是因为kafka请求访问这个路径,但是这个路径还不存在,zookeeper就抛了这个error。kafka会创建这个topic,然后访问zookeeper里topic对应的路径,zookeeper日志抛出erro
启动kafka
bin/kafka-server-start.sh config/server.properties
bin/kafka-server-start.sh -daemon config/server.properties
查看
ps aux | grep kafka
连接不上可能是超时时间太短
需要先搭建zookeeper集群
修改zookeeper配置文件
vim zoo.cfg
# 设置
dataDir=/opt/zookeeper/data
# 添加
server.1=node2:2881:3881
server.2=node3:2881:3881
server.3=node4:2881:3881
再搭建kafka集群
配置环境变量:KAFKA_HOME=/opt/kafka
PATH=$PATH:$KAFKA_HOME/bin
配置node2
vim /opt/kafka/config/server.properties//修改配置文件
broker.id=0
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://node2:9092 //advertised(广告),会写到zookeeper里去的发布地址
log.dirs=/opt/kafka/kafka-logs
zookeeper.connect=node2:2181,node3:2181,node4:2181/myKafka //三个节点下的/myKafka
配置node3
vim /opt/kafka/config/server.properties
broker.id=1
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://node3:9092
log.dirs=/opt/kafka/kafka-logs
zookeeper.connect=node2:2181,node3:2181,node4:2181/myKafka
node4配置
vim /opt/kafka/config/server.properties
broker.id=2
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://node4:9092
log.dirs=/opt/kafka/kafka-logs
zookeeper.connect=node2:2181,node3:2181,node4:2181/myKafka
启动kafka
[root@node2 ~]# kafka-server-start.sh /opt/kafka/config/server.properties
[root@node3 ~]# kafka-server-start.sh /opt/kafka/config/server.properties
[root@node4 ~]# kafka-server-start.sh /opt/kafka/config/server.properties
绑定了zookeeper集群,kafka会自动形成集群,向zookeeper中/myKafka创建/controller节点(节点名唯一),谁创建成功谁就是主节点。
Kafka-eagle可以发邮件提醒运维人员
网友评论