Apache Kafka
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. This blueprint implements Kafka using KRaft (Kafka Raft) mode, a significant architectural improvement that eliminates the need for Zookeeper.
Why and what's KRaft?
KRaft mode simplifies Kafka deployments, enhances scalability, and improves overall system performance. By using a built-in consensus protocol, KRaft reduces operational complexity, potentially speeds up broker startup times, and allows for better handling of metadata operations. This architectural shift enables Kafka to manage larger clusters more efficiently, making it an attractive option for organizations looking to streamline their event streaming infrastructure and prepare for future scalability needs.
Strimzi for Apache Kafka
Strimzi provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations. Strimzi combines security and simple configuration to deploy and manage Kafka on Kubernetes using kubectl and/or GitOps based on the Operator Pattern.
As of version 0.32.0, Strimzi provides full support for deploying Kafka clusters using KRaft, making it easier for organizations to leverage this new architecture. By using Strimzi, you can seamlessly deploy and manage Kafka clusters in KRaft mode on Kubernetes, taking advantage of its custom resource definitions (CRDs) and operators to handle the complexities of configuration and lifecycle management.
Architecture
Architecture diagram work in progress