Kafka for Administrators Training Course

Kafka Administration Essentials

• The role of Kafka in modern data platforms and standard production responsibilities.
• Key operational concepts: brokers, topics, partitions, offsets, and consumer groups.
• Fundamentals of replication, including leaders, followers, in-sync replicas, and availability trade-offs.
• High-level operational insights and terminology commonly found in runbooks.

KRaft Mode and Cluster Design

• KRaft fundamentals: controllers, metadata quorum, elections, and their operational significance.
• Deployment planning, focusing on sizing for throughput, partition counts, retention policies, and growth.
• Node roles and configurations, such as combined versus dedicated controllers and fault domain considerations.
• Practical lab: inspect KRaft metadata, validate quorum health, and analyze controller logs.

Installation, Configuration, and Daily Operations

• Installation methods (packages, tarballs, containers) and best practices for standardizing enterprise environments.
• Critical broker settings that affect reliability: listeners, replication, log directories, and retention.
• Safe operational procedures: startup sequences, graceful shutdowns, and validation checks.
• Practical lab: deploy a multi-node cluster, verify broker registration, and test basic produce/consume operations.

Managing Topics, Partitions, and Data Placement

• Managing the topic lifecycle via the Kafka CLI: creation, description, configuration updates, and deletion.
• Selecting appropriate partition and replication factors for real-world workloads, while avoiding common anti-patterns.
• Partition reassignment and balancing strategies, including when to move partitions and how to monitor progress.
• Practical lab: create topics, initiate partition reassignments, simulate a broker failure, and verify recovery.

Securing Kafka for Production

• Implementing TLS for client and inter-broker traffic, covering certificates, trust chains, and validation.
• Authentication using SASL, including mechanism selection and avoiding common misconfigurations.
• Authorization via ACLs, focusing on least-privilege patterns for admins, producers, and consumers.
• Practical lab: enable TLS and SASL, verify client connections, and apply role-based ACLs.

Observability, Reliability, and Troubleshooting

• Key monitoring areas: controller health, under-replicated partitions, request latency, and resource saturation.
• Analyzing logs and metrics, including reading broker logs and exporting metrics via JMX for observability stacks.
• Operational playbooks for rolling restarts, safe configuration changes, and managing disk-space or ISR issues.
• Practical lab: define a minimal alert set, diagnose a degraded cluster, and restore healthy replication.

Upgrades and Disaster Recovery Readiness

• Kafka upgrade planning, including compatibility checks, staging, and rollback strategies.
• Backup and recovery expectations, clarifying what can be backed up and how to recover configurations.
• Overview of cross-cluster replication and the use of MirrorMaker 2 for disaster recovery and migrations.
• Course conclusion: operational checklists, handover documentation, and next steps for production deployment.