Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Architecture and CUDA Fundamentals

• Distinct architectural differences between CPUs and GPUs.
• The NVIDIA GPU streaming multiprocessor architecture.
• An overview of the CUDA programming paradigm.
• Heterogeneous computing and the host-device interaction model.

Establishing the CUDA Development Environment

• Setup and installation of CUDA Toolkit 13.x.
• Utilizing the NVCC compiler and managing the build process.
• Environment verification through device query mechanisms.
• Integration with IDEs and associated development utilities.

Developing and Executing CUDA Kernels

• Syntax for kernel functions and specific qualifiers.
• Configuring and executing kernel launches.
• Implementing vector addition and fundamental data-parallel structures.
• Implementing robust CUDA error-checking macros.

CUDA Thread Organization and Execution

• Structure of grids, blocks, and individual threads.
• Thread indexing techniques and global ID derivation.
• Warp execution dynamics and the SIMT model.
• Managing occupancy and resource efficiency.

GPU Memory Structure and Control

• Overview of memory types: global, shared, constant, and registers.
• Device memory allocation and deallocation procedures.
• Data transfer between host and device.
• Leveraging shared memory for intra-block cooperation.

Unified Memory and Data Movement

• The unified memory model and managed allocation strategies.
• Page migration mechanics and on-demand paging.
• Utilizing asynchronous prefetching via cudaMemPrefetchAsync.
• Applying memory advice hints to optimize access patterns.

System-Level Profiling via Nsight Systems

• Analyzing timelines in Nsight Systems.
• Pinpointing CPU-GPU synchronization events.
• Visualizing kernel execution flows and memory transactions.
• Interpreting system-wide performance metrics.

Kernel Refinement using Nsight Compute

• Interactive kernel profiling with Nsight Compute.
• Evaluating memory throughput and bandwidth efficiency.
• Analyzing compute utilization and warp state statistics.
• Applying guided analysis and optimization guidelines.

Concurrent Streams and Asynchronous Execution

• Understanding CUDA streams and the default stream behavior.
• Overlapping kernel execution with data transfer operations.
• Stream synchronization mechanisms and CUDA events.
• Design patterns for multi-stream pipelines.

Error Management and Debugging Utilities

• Interpreting CUDA API error codes and implementing recovery logic.
• Using Compute-sanitizer for memory access validation.
• Debugging kernels with cuda-gdb.
• Employing assertions and synchronous error detection methods.

Optimization Strategy Based on Profiling Data

• Methodology for iterative profiling.
• Identifying and prioritizing performance bottlenecks.
• Conducting performance regression tests.
• Documenting optimization choices and outcomes.

Comprehensive GPU-Accelerated Application Project

• Architecting a full-scale GPU-accelerated solution.
• Integrating profiling throughout the development lifecycle.
• Performance benchmarking and result reporting.
• Considerations for deploying in production environments.