主要内容来自

https://www.tutorialspoint.com/apache_kafka/apache_kafka_quick_guide.htm

Step 1: 安装并启动 zookeeper

A)  到 http://www-us.apache.org/dist/zookeeper/zookeeper-3.4.10/ 下载 zookeeper-3.4.10.tar.gz  到/opt/.

B) tar -zxf zookeeper-3.4.10.tar.gz

C)  修改 conf/zoo.cfg 使其包含如下设置

tickTime=2000

dataDir=/path/to/zookeeper/data

clientPort=2181

initLimit=5

syncLimit=2

D) 启动zookeeper服务器

 bin/zkServer.sh  start

E) (在另一个新窗口）启动zookeeper客户端

bin/zkCli.sh

Step 2：安装并启动kafka

A) 到 https://kafka.apache.org/downloads/  下载 kafka_2.11-0.11.0.1.tgz

B) 到/opt/运行 tar -zxf kafka_2.11-0.11.0.1.tgz

C) 在  /opt/kafka_2.11-0.11.0.1/中 运行

bin/kafka-server-start.sh config/server.properties

Step 3: SingleNode_SingleBroker 模式

A) Create a topic: (创建一个topic Hello-Kafka)  在/opt/kafka_2.11-0.11.0.1/中 运行 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor1 --partitions1 --topic Hello-Kafka

B) 在/opt/kafka_2.11-0.11.0.1/中 运行 bin/kafka-topics.sh --list --zookeeper localhost:2181 查看 topic list

C) (在另一个窗口中启动producer 产生messages): 在/opt/kafka_2.11-0.11.0.1/中 运行 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Hello-Kafka

D) (在另一个窗口中启动consumer来消费messages): 在/opt/kafka_2.11-0.11.0.1/中 运行

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic Hello-Kafka --from-beginning

E) 运行了C) and D)，then whatever messages you input in the terminal of C) will appear in the terminal of D).

Step 4: SingleNode_MultipleBrokers模式

A)  在kafka/config/中复制文件server.properties为server-one.properties和server-two.properties。在server-one.properties中包含如下内容

port.id=1

listeners=PLAINTEXT://:9093

log.dirs=/tmp/kafka-logs-1

在server-two.properties中包含如下内容

port.id=2

listeners=PLAINTEXT://:9094

log.dirs=/tmp/kafka-logs-2

B) 然后可以启动三个brokers:

Broker1

bin/kafka-server-start.sh   config/server.properties

Broker2

bin/kafka-server-start.sh config/server-one.properties

Broker3

bin/kafka-server-start.sh config/server-two.properties

C) 创建一个topic: 另开一个窗口，到/opt/kafka_2.11-0.11.0.1/中运行

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic Multibrokerapplication

D) The 'Describe' command is used to check which broker is listening on the current created topic as shown below −

bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic Multibrokerapplication

运行的结果

Topic:Multibrokerapplication PartitionCount:1 ReplicationFactor:3 Configs:

Topic: Multibrokerapplication Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1

From the above output, we can conclude that first line gives a

summary of all the partitions, showing topic name, partition count and

the replication factor that we have chosen already. In the second line,

each node will be the leader for a randomly selected portion of the

partitions.

In our case, we see that our third broker (with broker.id 2) is the leader. Then Replicas:2,0,1 means that all the brokers replicate the topic finallyIsris the set ofin-syncreplicas. Well, this is the subset of replicas that are currently alive and caught up by the leader.

E) 启动一个producer产生message:

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Multibrokerapplication

F) 启动一个Consumer接收消费message:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic Multibrokerapplication --from-beginning

关于topic的基本操作

A) 修改topic (Modifying a Topic)

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic Hello-kafka --partitions 2

这条命令修改Hello-kafka的分区为2。

B) 删除一个topic

bin/kafka-topics.sh --zookeeper localhost:2181  --delete  --topic Multibrokerapplication

Kafka-Python API

https://github.com/dpkp/kafka-python

用pip install kafka-python

安装kafka-python后， 启动zookeeper 和kafka server and client以后,可以测试 github的 kafka-python 中的例子。

Another python API:

https://github.com/confluentinc/confluent-kafka-python

But I have not tested the latter one.