Accelerated C++ Programming with CUDA Training Course

Introduction to GPU-Accelerated Computing

• Heterogeneous computing concepts and CPU-GPU architectural design
• CUDA execution models and memory hierarchies
• Compiling CUDA C++ using nvcc and CMake
• Validating the GPU development environment

Parallel Algorithms via Thrust and CUB

• GPU-optimized sorting, reduction, and transformation operations
• Adapting STL algorithms for GPU execution
• Utilizing Thrust device vectors and execution policies
• Leveraging CUB device-wide primitives for bespoke pipelines

GPU Memory Architecture and Management

• Distinguishing global, constant, and texture memory types
• Explicit device memory allocation and data transfer
• Employing Unified Memory to simplify data access
• Optimizing memory coalescing and access patterns

Asynchronous Execution via CUDA Streams

• Creation and administration of CUDA streams
• Overlapping kernel execution with data transfer operations
• Managing dependencies using CUDA events
• Tuning stream priorities and concurrency levels

Developing Custom CUDA Kernels

• Understanding the SIMT programming model and warp execution
• Configuring kernel launches and implementing grid-stride loops
• Thread indexing strategies and multidimensional grids
• Error handling and verification via the CUDA runtime API

Thread Hierarchy and Execution Model

• Structuring grids, blocks, and threads in device code
• Utilizing warp-level primitives and ballot operations
• Implementing block-level synchronization and barriers
• Analyzing occupancy and resource utilization

Cooperative Groups for Flexible Parallelism

• Using the cooperative_groups API and group types
• Implementing thread-block tiles and tiled_partition
• Executing grid-level cooperative launches
• Patterns for multi-grid synchronization

Shared Memory Optimization Strategies

• Managing shared memory banks and avoiding bank conflicts
• Applying tiling strategies to matrix operations
• Using shared memory as a user-managed cache
• Employing cuda::shared_memory_mdspan for multidimensional views

Kernel Fusion and Advanced Parallel Patterns

• Fusing multiple kernels to minimize launch overhead
• Implementing scan, reduce-by-key, and segmented algorithms
• Utilizing atomic operations and lock-free data structures
• Enhancing throughput with warp-aggregated atomics

Profiling and Optimization via Nsight Systems

• Analyzing CPU and GPU activity through timeline views
• Detecting memory transfer bottlenecks
• Profiling kernel performance and occupancy
• Iterative optimization using Nsight Compute

Modern C++ Features in CUDA Device Code

• Using Lambdas, constexpr, and auto within kernels
• Implementing C++17 parallel algorithms and execution policies
• Applying C++20 concepts and ranges on the device
• Leveraging C++23 support in nvcc and CCCL 3.x

CUDA Graphs and Advanced Asynchrony

• Definition and execution of CUDA graphs
• Capturing graphs from stream execution
• Updating graphs and utilizing conditional execution nodes
• Reducing launch latency for iterative workloads

Integration Patterns for Existing Applications

• Encapsulating GPU code behind C++ interfaces
• Managing multi-GPU and NUMA system configurations
• Integrating build systems with CMake and CUDA
• Debugging device code using cuda-gdb

Summary and Best Practices

• Selecting among Thrust, CUB, or custom kernels
• Ensuring performance portability across GPU architectures
• Organizing code and applying RAII for CUDA resources
• Next steps and advanced CUDA learning pathways