Skip to content
Related Articles

Related Articles

Difference Between Apache Kafka and Apache Flume

Improve Article
Save Article
  • Difficulty Level : Easy
  • Last Updated : 17 May, 2020
Improve Article
Save Article

Apache Kafka: It is an open-source stream-processing software platform written in Java and Scala. It is made by LinkedIn which is given to the Apache Software Foundation. Apache Kafka aims to provide a high throughput, unified, low-latency platform for handling the real-time data feeds. Kafka generally used TCP based protocol which optimized for efficiency. It is very fast and performs 2 million writes per second.
It also guarantees zero percent data loss.
Apache Kafka generally used for real-time analytics, ingestion data into the Hadoop and to spark, error recovery, website activity tracking.

Flume: Apache Flume is a reliable, distributed, and available software for efficiently aggregating, collecting, and moving large amounts of log data. It has a flexible and simple architecture based on streaming data flows. It is written in java. It has its own query processing engine which makes it to transform each new batch of data before it is moved to the intended sink. It has a flexible design.

Below is a table of differences between Apache Kafka and Apache Flume:

Apache KafkaApache Flume
Apache Kafka is a distributed data system.Apache Flume is a available, reliable, and distributed system.
It is optimized for ingesting and processing streaming data in real-time.It is efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store.
It is basically working as a pull model.It is basically working as a push model .
It is easy to scale.It is not scalable in comparison with Kafka.
An fault-tolerant, efficient and scalable messaging system.It is specially designed for Hadoop.
It supports automatic recovery if resilient to node failure.You will lose events in the channel in case of flume-agent failure.
Kafka runs as a cluster which handles the incoming high volume data streams in the real time.Flume is a tool to collect log data from distributed web servers.
Kafka will treat each topic partition as an ordered set of messages.Flume can take in streaming data from the multiple sources for storage and analysis which use in Hadoop.
My Personal Notes arrow_drop_up
Related Articles

Start Your Coding Journey Now!