To study Apache Kafka Architecture in details, and how to install, deploy configure Apache kafka.

ashwintelmore

Ashwin Telmore

Posted on November 17, 2022

To study Apache Kafka Architecture in details, and how to install, deploy configure Apache kafka.

Apache Kafka is an open-source, distributed event streaming platform developed by the Apache Software Foundation. This is written in Scala and Java programming languages. You can install Kafka on any platform that supports Java programming language.

This tutorial provides you with step-by-step instructions to install Apache Kafka on Ubuntu 22.04 LTS Linux system. You will also learn to create topics in Kafka and run producer and consumer nodes.

Prerequisites
You must have sudo privileged account access to the Ubuntu 22.04 Linux system.

Step 1 – Installing Java
We can run the Apache Kafka server on systems that support Java. So make sure, you have Java installed on your Ubuntu system.

Use the following commands to install OpenJDK on your Ubuntu system from the official repositories.

sudo apt update  
sudo apt install default-jdk 
Enter fullscreen mode Exit fullscreen mode

Verify the current active Java version.

java --version 
Enter fullscreen mode Exit fullscreen mode

Output:

openjdk version "11.0.15" 2022-04-19
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1)
OpenJDK 64-Bit Server VM (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1, mixed mode, sharing)

Step 2 – Download Latest Apache Kafka
You can download the latest Apache Kafka binary files from its official download page. Alternativaly you can download Kafka 3.2.0 with the below command.

wget https://dlcdn.apache.org/kafka/3.2.0/kafka_2.13-3.2.0.tgz 
Enter fullscreen mode Exit fullscreen mode

Then extract the downloaded archive file and place them under /usr/local/kafka directory.


tar xzf kafka_2.13-3.2.0.tgz 
sudo mv kafka_2.13-3.2.0 /usr/local/kafka 

Enter fullscreen mode Exit fullscreen mode

Step 3 – Create Systemd Startup Scripts
Now, create systemd unit files for the Zookeeper and Kafka services. That will help you to start/stop the Kafka service in an easy way.

First, create a systemd unit file for Zookeeper:

sudo nano /etc/systemd/system/zookeeper.service 
Enter fullscreen mode Exit fullscreen mode

And add the following content:

[Unit]
Description=Apache Zookeeper server
Documentation=http://zookeeper.apache.org
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
ExecStart=/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties
ExecStop=/usr/local/kafka/bin/zookeeper-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Save the file and close it.

Next, create a systemd unit file for the Kafka service:

sudo nano /etc/systemd/system/kafka.service 
Enter fullscreen mode Exit fullscreen mode

Add the below content. Make sure to set the correct JAVA_HOME path as per the Java installed on your system.

find /usr/lib/jvm/java-1.x.x-openjdk

sudo vim /etc/profile
Enter fullscreen mode Exit fullscreen mode
export JAVA_HOME="path that you found"
export PATH=$JAVA_HOME/bin:$PATH
Enter fullscreen mode Exit fullscreen mode

logout and login again, reboot, or use source /etc/profile to apply changes immediately in your current shell

[Unit]
Description=Apache Kafka Server
Documentation=http://kafka.apache.org/documentation.html
Requires=zookeeper.service

[Service]
Type=simple
Environment="JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64"
ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Save the file and close.

Reload the systemd daemon to apply new changes.

sudo systemctl daemon-reload 
Enter fullscreen mode Exit fullscreen mode

This will reload all the systemd files in the system environment.

Step 4 – Start Zookeeper and Kafka Services

Let’s start both services one by one. First, you need to start the ZooKeeper service and then start Kafka. Use the systemctl command to start a single-node ZooKeeper instance.

sudo systemctl start zookeeper 
sudo systemctl start kafka 
Enter fullscreen mode Exit fullscreen mode

Verify both of the services status:

sudo systemctl status zookeeper 
sudo systemctl status kafka 
Enter fullscreen mode Exit fullscreen mode

That’s it. You have successfully installed the Apache Kafka server on Ubuntu 22.04 system. Next, we will create topics in the Kafka server.

💖 💪 🙅 🚩
ashwintelmore
Ashwin Telmore

Posted on November 17, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related