Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Foundations of Data-Intensive Platform Engineering
- Introduction to data-intensive application architectures
- Key challenges in big data platform engineering
- Overview of data processing architectural patterns
Data Modeling and Management
- Principles for scalable data modeling
- Evaluating data storage options and optimization strategies
- Managing the data lifecycle within distributed systems
Big Data Processing Frameworks
- Survey of major processing tools (Hadoop, Spark, Flink)
- Distinctions between batch and stream processing
- Constructing end-to-end big data processing pipelines
Real-Time Analytics Platforms
- Architecting systems for real-time analytics
- Stream processing engines (Kafka Streams, Apache Storm)
- Creating real-time dashboards and visual interfaces
Data Pipeline Orchestration
- Workflow management using Apache Airflow and similar tools
- Automating pipelines to enhance operational efficiency
- Implementing monitoring and alerting mechanisms for pipelines
Platform Security and Compliance
- Best practices for securing data platforms
- Ensuring data privacy and regulatory adherence
- Deploying secure data access control mechanisms
Performance Tuning and Optimization
- Strategies for optimizing data throughput and reducing latency
- Scaling approaches for data-intensive environments
- Conducting performance benchmarking and continuous monitoring
Case Studies and Best Practices
- Analyzing successful data platform implementations
- Insights and lessons from industry leaders
- Exploring emerging trends in platform engineering
Capstone Project
- Designing a comprehensive platform solution for a data-intensive use case
- Building a functional prototype of the data processing pipeline
- Evaluating the platform’s performance and scalability
Summary and Next Steps
Requirements
- Solid grasp of fundamental data structures and algorithms
- Practical experience with Java, Scala, or Python
- Knowledge of core database concepts and SQL
Target Audience
- Software developers
- Data engineers
- Technical leads
21 Hours
Testimonials (2)
the overall concept of DevOps/DevSecOps is not a team but a culture, a mind set
Mpho Mashele - South Africa Reserve Bank
Course - DevOps and Platform Engineering: A Collaborative Approach
About the microservices and how to maintenance kubernetes