Apache Kafka [Kraft]
Install Java JDK version 11
1.Install Java JDK version 11 :
wget -O- https://apt.corretto.aws/corretto.key | sudo apt-key add -
sudo add-apt-repository 'deb https://apt.corretto.aws stable main'
sudo apt-get update; sudo apt-get install -y java-11-amazon-corretto-jdk
If see add-apt-repository: command not found error :
sudo apt-get install software-properties-common
2.Upon completion, you should get a similar output when doing java -version :
openjdk version "11.0.10" 2021-01-19 LTS
OpenJDK Runtime Environment Corretto-11.0.10.9.1 (build 11.0.10+9-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.10.9.1 (build 11.0.10+9-LTS, mixed mode)shell
Install Kafka on Debian
1.Download the latest version of Apache Kafka. For example : Scala 2.13 - kafka_2.13-3.5.1.tgz (asc, sha512) :
https://kafka.apache.org/downloads
wget https://downloads.apache.org/kafka/3.5.1/kafka_2.13-3.5.1.tgz
2.Extract kafka :
tar xvfz kafka_2.13-3.5.1.tgz
mv kafka_2.13-3.5.1 kafka
mv kafka /usr/local/bin
3.Create kafka usre :
useradd kafka -s /sbin/nologin
4.Change owner kafka directory in /usr/local/bin :
cd /usr/local/bin
chown -R kafka:kafka kafka
Start Kafka
1.Generate a new ID for your cluster :
kafka/bin/kafka-storage.sh random-uuid
This returns a UUID, for example 69BLMI7rP_ql1KjfKs8L9Z
2.Format your storage directory (replace <uuid> by your UUID) :
kafka/bin/kafka-storage.sh format -t <uuid> -c kafka/config/kraft/server.properties
3.Move Kraft config to /etc :
mv kafka/config/kraft /etc/
4.Change owner kraft directory in /etc :
cd /etc
chown -R kafka:kafka kraft
5.Change log.dirs :
vim /etc/kraft/server.properties
log.dirs=/usr/local/bin/kafka/tmp/kraft-combined-logs
6.Change owner kraft logs directory :
cd /usr/local/bin/kafka/
chown -R kafka:kafka tmp
Create Kafka service
1.Edit /usr/lib/systemd/system/kafka.service :
[Unit]
Description=Apache Kafka server [kraft]
Documentation=https://zakops.com/
[Service]
Type=simple
User=kafka
Group=kafka
Environment=JAVA_HOME=/usr/lib/jvm/java-11-amazon-corretto
ExecStart=/usr/local/bin/kafka/bin/kafka-server-start.sh /etc/kraft/server.properties
ExecStop=/usr/local/bin/kafka/bin/kafka-server-stop.sh /etc/kraft/server.properties
[Install]
WantedBy=multi-user.target
2.Reload and start the systemd services :
systemctl daemon-reload
systemctl start kafka.service