- \n
- Ubuntu 18.04<\/li>\n
- Root privileges<\/li>\n<\/ul>\n
What we will do?<\/h2>\n
- \n
- Install Java OpenJDK 8<\/li>\n
- Install Apache Zookeeper<\/li>\n
- Download and Configure Apache Kafka<\/li>\n
- Configure Apache Kafka and Zookeeper as s Service<\/li>\n
- Testing<\/li>\n<\/ol>\n
Step 1 – Install Java OpenJDK 8<\/h2>\n
Apache Kafka has been written in Java and Scala, so we need to install java on the server.<\/p>\n
Before installing any packages, update the repository and upgrade all packages.<\/p>\n
sudo apt update
sudo apt upgrade<\/p>\nNow install the Java OpenJDK 8 from the Ubuntu repository using the apt command below.<\/p>\n
sudo apt install openjdk-8-jdk -y<\/p>\n
After the installation is complete, check the java installed version.<\/p>\n
java -version<\/p>\n
Now you will see the java OpenJDK 8 installed on Ubuntu 18.04.<\/p>\n
![\"Install](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu.png\" "\\"\\"")
<\/a><\/p>\nStep 2 – Install Apache Zookeeper<\/h2>\n
Apache Kafka uses zookeeper for the electing controller, cluster membership, and topics configuration. Zookeeper s a distributed configuration and synchronization service.<\/p>\n
In this step, we will install Zookeeper from the Ubuntu repository.<\/p>\n
Run the apt command below.<\/p>\n
sudo apt install zookeeperd -y<\/p>\n
Wait until the installation is complete.<\/p>\n
![\"Install](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-1.png\" "\\"\\"")
<\/a><\/p>\nStep 3 – Download and Configure Apache Kafka<\/h2>\n
In this step, we will install the Apache Kafka using the binary files that can be downloaded from the Kafka website. We will install and configure apache Kafka and run it as a non-root user.<\/p>\n
Add a new user named ‘kafka’.<\/p>\n
useradd -d \/opt\/kafka -s \/bin\/bash kafka
passwd kafka<\/p>\nNow go to the ‘\/opt’ directory and download the Apache Kafka binary files using wget.<\/p>\n
cd \/opt
wget http:\/\/www-eu.apache.org\/dist\/kafka\/2.0.0\/kafka_2.11-2.0.0.tgz<\/p>\nNow create a new kafka directory.<\/p>\n
mkdir -p \/opt\/kafka<\/p>\n
Extract the kafka_*.tar.gz file to the ‘kafka’ directory and change the owner of directory to the ‘kafka’ user and group.<\/p>\n
tar -xf kafka_2.11-2.0.0.tgz -C \/opt\/kafka –strip-components=1
sudo chown -R kafka:kafka \/opt\/kafka<\/p>\nNow login to the ‘kafka’ user and edit the server.properties configuration.<\/p>\n
su – kafka
vim config\/server.properties<\/p>\nPaste the following configuration to the end of the line.<\/p>\n
delete.topic.enable = true<\/pre>\n
Save and exit.<\/p>\n
![\"Start](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-2.png\" "\\"\\"")
<\/a><\/p>\nThe Apache Kafka configuration has been completed.<\/p>\n
Step 4 – Configure Apache Kafka and Zookeeper as Services<\/h2>\n
In this step, we will configure the Apache Kafka as a service and configure the customs service configuration for the zookeeper.<\/p>\n
Go to the ‘\/lib\/systemd\/system’ directory and create a new service file ‘zookeeper.service’.<\/p>\n
cd \/lib\/systemd\/system\/
vim zookeeper.service<\/p>\nPaste the configuration below.<\/p>\n
[Unit] \nRequires=network.target remote-fs.target \nAfter=network.target remote-fs.target \n \n[Service] \nType=simple \nUser=kafka \nExecStart=\/opt\/kafka\/bin\/zookeeper-server-start.sh \/opt\/kafka\/config\/zookeeper.properties \nExecStop=\/opt\/kafka\/bin\/zookeeper-server-stop.sh \nRestart=on-abnormal \n \n[Install] \nWantedBy=multi-user.target<\/pre>\n
Save and exit.<\/p>\n
Now create the Apache Kafka service file ‘kafka.service’.<\/p>\n
vim kafka.service<\/p>\n
Paste the configuration below.<\/p>\n
[Unit] \nRequires=zookeeper.service \nAfter=zookeeper.service \n \n[Service] \nType=simple \nUser=kafka \nExecStart=\/bin\/sh -c '\/opt\/kafka\/bin\/kafka-server-start.sh \/opt\/kafka\/config\/server.properties' \nExecStop=\/opt\/kafka\/bin\/kafka-server-stop.sh \nRestart=on-abnormal \n \n[Install] \nWantedBy=multi-user.target<\/pre>\n
Save and exit.<\/p>\n
Reload the systemd manager configuration.<\/p>\n
systemctl daemon-reload<\/p>\n
Now start Apache Zookeeper and Apache Kafka services.<\/p>\n
systemctl start zookeeper
systemctl enable zookeeper<\/p>\nsystemctl start kafka
systemctl enable kafka<\/p>\n![\"Start](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-3.png\" "\\"\\"")
<\/a><\/p>\nThe apache zookeeper and Kafka are up and running.<\/p>\n
Zookeeper running under port ‘2181’, and Kafka on port ‘9092’, check it using the\u00a0netstat<\/a> command below.<\/p>\n
netstat -plntu<\/p>\n
![\"Software](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-4.png\" "\\"\\"")
<\/a><\/p>\nStep 5 – Testing Apache Kafka<\/h2>\n
Login to the ‘kafka’ user and go to the ‘bin\/’ directory.<\/p>\n
su – kafka
cd bin\/<\/p>\nNow create a new topic named ‘HakaseTesting’ using the ‘kafka-topics.sh’ executable file.<\/p>\n
.\/kafka-topics.sh –create –zookeeper localhost:2181 \\
–replication-factor 1 –partitions 1 \\
–topic HakaseTesting<\/p>\nAnd run the ‘kafka-console-producer.sh’ with the ‘HakaseTesting’ topic.<\/p>\n
.\/kafka-console-producer.sh –broker-list localhost:9092 \\
–topic HakaseTesting<\/p>\nNow open a new terminal and log in to the server, then login to the ‘kafka’ user.<\/p>\n
Run ‘kafka-console-consumer.sh’ for the ‘HakaseTesting’ topic.<\/p>\n
.\/kafka-console-consumer.sh –bootstrap-server localhost:9092 \\
–topic HakaseTesting –from-beginning<\/p>\nAnd when you type any input from the ‘kafka-console-producer.sh’ shell, you will get the same result on the ‘kafka-console-consumer.sh’ shell.<\/p>\n
![\"Testing](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-5.png\" "\\"\\"")
<\/a><\/p>\nThe installation and configuration for Apache Kafka on Ubuntu 18.04 has been completed successfully.<\/p>\n
Reference<\/h2>\n
\nAbout Muhammad Arul<\/strong><\/p>\n
Muhammad Arul is a freelance system administrator and technical writer. He is working with Linux Environments for more than 5 years, an Open Source enthusiast and highly motivated on Linux installation and troubleshooting. Mostly working with RedHat\/CentOS Linux and Ubuntu\/Debian, Nginx and Apache web server, Proxmox, Zimbra Administration, and Website Optimization. Currently learning about OpenStack and Container Technology.<\/p>\n<\/div>\n
\nShare this page:<\/b><\/p>\n
\n
![\"\"](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-6.png\" "\\"\\"")
<\/a>
\n![\"\"](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-7.png\" "\\"\\"")
<\/a>
\n![\"\"](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-8.png\" "\\"\\"")
<\/a>
\n![\"\"](\"https:\/\/afaghhosting.net\/blog\/wp-content\/uploads\/2018\/10\/how-to-install-apache-kafka-distributed-streaming-platform-on-ubuntu-9.png\" "\\"\\"")
<\/a>\n<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"Apache Kafka is a distributed streaming platform developed by Apache Software Foundation and written in Java and Scala. Apache Kafka was originally developed by LinkedIn, and was open sourced in 2011. Apache Kafka is used for building real-time streaming data pipeline that reliably gets data between system and applications. It provides a unified, high-throughput, and …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36],"tags":[],"class_list":["post-6967","post","type-post","status-publish","format-standard","hentry","category-36"],"_links":{"self":[{"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/posts\/6967","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/comments?post=6967"}],"version-history":[{"count":0,"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/posts\/6967\/revisions"}],"wp:attachment":[{"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/media?parent=6967"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/categories?post=6967"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/afaghhosting.net\/blog\/wp-json\/wp\/v2\/tags?post=6967"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"Install](\"https:\/\/www.howtoforge.com\/images\/how_to_install_apache_kafka_on_ubuntu_1804\/big\/1.png\" "\\"\\"")
<\/a><\/p>\n