How to Create Apache Kafka Producer with Conduktor?

Last Updated : 01 Aug, 2022

Kafka Producers are going to write data to topics and topics are made of partitions. Now the producers in Kafka will automatically know to which broker and partition to write based on your message and in case there is a Kafka broker failure in your cluster the producers will automatically recover from it which makes Kafka resilient and which makes Kafka so good used today. So if we look at a diagram to have the data in our topic partitions we’re going to have a producer on the left-hand side sending data into each of the partitions of our topics.

Create Apache Kafka Producer with Conduktor

So how does a producer know how to send the data to a topic partition? For this, we can use Message Keys. So alongside the message value, we can choose to send a message key and that key can be whatever you want it could be a string, it could be a number whatever you want and it turns out that if you don’t send the key, the key is set to null then the data will be sent in a Round Robin fashion to make it very simple. So that means that your first message is going to be sent to partition 0, and then your second message to partition 1 and then partition 2, and so on. This is why it’s called Round Robin, but in case you send a key with your message, all the messages that share the same key will always go to the same partition. So this is a very very important property of Kafka.

Note: Please read this article, Apache Kafka Producer, to know more about Kafka Producer.

So in this article, we are going to discuss how to Create Apache Kafka Producer with Conduktor. Talking briefly about Conduktor, Conduktor is a full-featured native desktop application that plugs directly into Apache Kafka to bring visibility to the management of Kafka clusters, applications, and microservices. It’s eventually helping companies make the most of their existing engineering resources, and minimizing the need for deep in-house expertise.

Step by Step Implementation

Step 1: Create your Topics in Conduktor Tool

Refer to this article, How to Create Kafka Topics using Conduktor Tool, and create your Topics in Conduktor Tool.

Step 2: Produce Data Without Key

Now go to the Producer tab as shown in the below image.

A pop-up will be shown up on your screen like the below image. Here go to the Data tab and then choose the Topic in which you want to Produce your messages. Let’s first produce some messages without any Key. Inside Value, you have to provide the message you want to publish. And finally, click on the Produce to Topic button and you can see on the right-hand side the message has been published successfully. Please refer to the below image if you are stuck somewhere.

If you click on the Produce to Topic button many times then the message will be published many times. So here you can see the partition number and the offset. Though we have not provided any key so the data is sent in a Round Robin fashion as we have discussed above.

Now if you see the size of our topic has been increased. Please compare it with the Step 2 screenshot.

You can also do the following stiff while Creating the Apache Kafka Producer with Conduktor

You can change the Serializer according to what you need
Generate some random data
Create a new template, etc.

Refer to the below image for the same and explore by yourself.

Step 3: Produce Data With Key

Now let’s Produce some Data With the Key. Let’s provide the key in the Key field and click on the Produce to Topic button. Here you can see all the data has been produced in the same partition as we have provided the Key, only Offset is different. Please refer to the below image if you are stuck somewhere.

Step 4: Produce Data Automatically

If you want to send the data Automatically you have to go to the Flow tab and you have to choose the automatic mode. You can also choose the time interval. Then click on the Start Producing button. Please refer to the below image if you are stuck somewhere.