Core concepts of Kafka
Chapter-2
1-Producer
2-Consumer
3-Broker
4-Cluster
5-Topic
6-Partitions
7-Offset
8-Consumer Groups
1-Producer- it is a application who send data/records/message/message records .in terms of kafka it simply called array of bytes.
2-Consumer -it is a application who receive or consume data/records/message/message records from broker
producer can’t send data directly to consumer and consumer can’t receive/consume data directly from producer.
3-Broker — it is a kafka server , it act like message broker b/w producer and consumer because producer and consumer not directly communicate .it use as agent for producer and consumer .
4-Cluster — it is a group of kafka server who act together for a common purpose .
5-Topic- it is a unique name for data stream , it can hold same category of data to easily identify .
6-Partitions- Topic breaks in smaller part its called Partitions.
when you are collecting serval terabyte of data then topic breaks into multiple application and distributed into multiple computers in kafka cluster .
7-Offset- it is a unique sequence id of every message in Partitions and it will automatically assigned .once id assigned it will never change (immutable).
8-Consumer Groups Consumer groups give Kafka the flexibility to have the advantages of both message queuing and publish-subscribe models. Kafka consumers belonging to the same consumer group share a group id. The consumers in a group then divides the topic partitions as fairly amongst themselves as possible by establishing that each partition is only consumed by a single consumer from the group.
