安装

安装JAVA环境

yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y  

下载安装包和解压

wget https://archive.apache.org/dist/kafka/3.4.0/kafka_2.12-3.4.0.tgz  
tar zxf kafka_2.12-3.4.0.tgz -C /usr/local  
cd  /usr/local && ln -s kafka_2.12-3.4.0/ kafka  

配置zookeeper

配置kafka_2.12-3.4.0/config下"zookeeper.properties"。修改dataDir和clientPort。前者是快照存放地址(自己随意配置)，后者是客户端连接zookeeper服务的端口。默认端口2181 最好默认不修改

dataDir=/data/zookeeper  
clientPort=2181  
maxClientCnxns=0  
admin.enableServer=false  

配置kafka

配置kafka_2.12-3.4.0/config下的“server.properties”，修改log.dirs和zookeeper.connect。前者是日志存放文件夹，后者是zookeeper连接地址（端口和clientPort保持一致）

broker.id=0  
listeners=PLAINTEXT://192.168.240.121:9092  
num.network.threads=3  
num.io.threads=8  
socket.send.buffer.bytes=102400  
socket.receive.buffer.bytes=102400  
socket.request.max.bytes=104857600  
log.dirs=/data/logs/kafka  
num.partitions=1  
num.recovery.threads.per.data.dir=1  
offsets.topic.replication.factor=1  
transaction.state.log.replication.factor=1  
transaction.state.log.min.isr=1  
log.retention.hours=168  
log.retention.check.interval.ms=300000  
zookeeper.connect=192.168.240.121:2181  
zookeeper.connection.timeout.ms=18000  
group.initial.rebalance.delay.ms=0  

kafka的相关命令

开启kafka自带zookeeper

# 前台运行
./bin/zookeeper-server-start.sh ./config/zookeeper.properties
# 后台运行
./bin/zookeeper-server-start.sh -daemon ./config/zookeeper.properties

开启kafka

# 前台运行
bin/kafka-server-start.sh config/server.properties

# 后台运行
./bin/kafka-server-start.sh -daemon ./config/server.properties

使用jps命令查看是否正常了

[root@ kafka]# jps
19397 Jps  
18424 QuorumPeerMain  
19273 Kafka

创建kafka主题

其中zookeeper 可以使用hostname 也可以使用IP,若使用hostname,请注意hosts 即/etc/hosts 与hostname最好保持一致

[root@ kafka]# ./bin/kafka-topics.sh --create --topic xm2-game-log --bootstrap-server 192.168.240.121:9092
Created topic xm2-game-log.  

显示kafka所有主题

./bin/kafka-topics.sh --list --zookeeper 192.168.240.121:2181

查看某个主题的详细信息

[root@tx-office-centos7-240-121 kafka]# ./bin/kafka-topics.sh  --bootstrap-server 192.168.240.121:9092 --describe  --topic xm2-game-log
Topic: xm2-game-log    TopicId: eSPFRNzBQHaWhfo0SslmXg PartitionCount: 1   ReplicationFactor: 1    Configs:  
    Topic: xm2-game-log Partition: 0    Leader: 0   Replicas: 0 Isr: 0

创建kafka生产者

./bin/kafka-console-producer.sh --broker-list 192.168.240.121:9092 --topic xm2-game-log

创建kafka消费者

./bin/kafka-console-consumer.sh --bootstrap-server 192.168.240.121:9092 --topic xm2-game-log --from-beginning