Course Outline
Introduction to GPU-Accelerated Computing
- Heterogeneous computing and CPU-GPU architecture
- CUDA execution and memory spaces
- Compiling CUDA C++ with nvcc and CMake
- Verifying the GPU development environment
Parallel Algorithms with Thrust and CUB
- GPU-accelerated sort, reduce, and transform operations
- Refactoring STL algorithms for GPU execution
- Thrust device vectors and execution policies
- CUB device-wide primitives for custom pipelines
GPU Memory Architecture and Management
- Global, constant, and texture memory types
- Explicit device memory allocation and transfers
- Unified Memory for simplified data access
- Memory coalescing and access pattern optimization
Asynchronous Execution with CUDA Streams
- Creating and managing CUDA streams
- Overlapping kernel execution with data transfers
- CUDA events for dependency management
- Stream priorities and concurrency tuning
Writing Custom CUDA Kernels
- The SIMT programming model and warp execution
- Kernel launch configuration and grid-stride loops
- Thread indexing and multidimensional grids
- Error handling and CUDA runtime API checks
Thread Hierarchy and Execution Model
- Grids, blocks, and threads in device code
- Warp-level primitives and ballot operations
- Block-level synchronization and barriers
- Occupancy and resource utilization analysis
Cooperative Groups for Flexible Parallelism
- The cooperative_groups API and group types
- Thread-block tiles and tiled_partition
- Grid-level cooperative launches
- Multi-grid synchronization patterns
Shared Memory Optimization Techniques
- Shared memory banks and avoiding bank conflicts
- Tiling strategies for matrix operations
- Using shared memory as a user-managed cache
- cuda::shared_memory_mdspan for multidimensional views
Kernel Fusion and Advanced Parallel Patterns
- Fusing multiple kernels to reduce launch overhead
- Scan, reduce-by-key, and segmented algorithms
- Atomic operations and lock-free data structures
- Warp-aggregated atomics for improved throughput
Profiling and Optimization with Nsight Systems
- Timeline analysis of CPU and GPU activity
- Identifying memory transfer bottlenecks
- Kernel performance and occupancy profiling
- Iterative optimization with Nsight Compute
Modern C++ Features in CUDA Device Code
- Lambdas, constexpr, and auto in kernels
- C++17 parallel algorithms and execution policies
- C++20 concepts and ranges on device
- C++23 support in nvcc and CCCL 3.x
CUDA Graphs and Advanced Asynchrony
- Defining and launching CUDA graphs
- Graph capture from stream execution
- Graph update and conditional execution nodes
- Reducing launch latency for iterative workloads
Integration Patterns for Existing Applications
- Wrapping GPU code behind C++ interfaces
- Managing multi-GPU and NUMA systems
- Build system integration with CMake and CUDA
- Debugging device code with cuda-gdb
Summary and Best Practices
- Choosing between Thrust, CUB, and custom kernels
- Performance portability across GPU architectures
- Code organization and RAII for CUDA resources
- Next steps and advanced CUDA learning paths
Requirements
- Fundamental C++ skills, including lambda expressions, templates, and the STL
- Understanding of standard algorithms, containers, and iterators
- Proficiency with loops, conditionals, and functions
- No prior experience with CUDA or GPU programming is required
Audience
- C++ developers aiming to accelerate compute-intensive applications with GPUs
- Software engineers transitioning from CPU-only development to heterogeneous parallel programming
- Performance engineers and quantitative developers handling large datasets
Testimonials (3)
Initially, the trainer's pace seemed a bit too fast for me, but after offering the feedback during the training, he acknowledged it and slowed down the pace without sacrificing any of the lectures. Good rapport with the audience, very friendly and open for discussions.
Alexandru Ostafi - Siemens
Course - Advanced C++ : Practical workshop
Detailed explanation, re-iteration of points in a quite subtle way that really drove the knowledge home very well. Rod's willingness to double-check the odd obscure question that we rasied, to be sure that his answers were 100% right. Also, his interest in discussing the pros & cons of alternate coding styles so that we learnt not only how to use C++ in our intended way, but why it should be done that way.
Nick Dillon - cellxica Ltd
Course - Using C++ in Embedded Systems - Applying C++11/C++14
Experience sharing, it's teacher's know-how and valuable.
