Confluent Apache Kafka: Cluster Operations and Configuration Training Course

Module 1: Introduction to Confluent Apache Kafka Architecture and Cluster Configuration

• Overview of Apache Kafka and the Confluent Platform.
• Core concepts including Kafka brokers, producers, consumers, and topics.
• The significance of Kafka in data streaming and messaging pipelines.
• Confluent-specific extensions: REST Proxy, Schema Registry, and Control Center.
• Kafka cluster architecture details, including brokers, partitions, and replication.

Module 2: Zookeeper Quorum Configuration

• Introduction to Zookeeper.
• The functional role of Zookeeper within a Kafka cluster.
• Determining appropriate Zookeeper quorum sizes.
• Configuring Zookeeper parameters.
• Implementing SSH on server infrastructure.
• Hands-on: Configuring Zookeeper (both as a team and as a service).
• Utilizing the Zookeeper Command Line Interface (CLI).
• Hands-on: Setting up the Zookeeper quorum.
• Understanding the internal file system of Zookeeper.
• Key performance factors influencing Zookeeper.
• Demo: Exploring Zookeeper management tools and Zoonavigator.

Module 3: Kafka Cluster Configuration

• Foundational Kafka concepts.
• Configuring Kafka instances.
• Hands-on: Configuring Kafka on a single broker.
• Hands-on: Executing standard Kafka commands.
• Hands-on: Establishing a Kafka multi-broker (cluster) setup.
• Hands-on: Conducting Kafka cluster tests.
• Verifying connectivity to the Kafka cluster.
• Key settings for the Advertised.listeners configuration.
• Managing topic configurations.
• Setting up message ingestion and delivery mechanisms within topics.
• Hands-on: Demonstrating Kafka resilience capabilities.
• Kafka performance considerations: I/O, network, RAM, CPU, and OS optimization.
• Additional Kafka performance factors.
• Hands-on: Modifying broker configurations effectively.

Module 4: Advanced Kafka Configuration

• Configuring Landoop Kafka Topics UI, Confluent REST Proxy, and Confluent Schema Registry.
• Sending and receiving messages via CLI, Java, and Spring Framework.
• Leveraging metrics and monitoring tools such as Confluent Control Center and Elasticsearch.
• Managing log files and offsets.
• Ensuring high availability and disaster recovery.
• Maintaining high availability through replication.
• Performance tuning for producers and consumers.
• Formulating disaster recovery strategies.
• Controlling failover processes and data recovery.
• Setting up connectors.
• Implementing Kafka Connect.
• Implementing Kafka security features.

Summary and Next Steps