GPU Programming with CUDA Training Course

Introduction

• An overview of CUDA
• Comparison of CUDA vs OpenCL vs SYCL
• Key features and architecture of CUDA
• Setting up the development environment

Getting Started

• Creating a new CUDA project in Visual Studio Code
• Examining the project structure and files
• Compiling and executing the program
• Displaying output using printf and fprintf

CUDA API

• The role of the CUDA API in host programs
• Querying device information and capabilities via the CUDA API
• Allocating and deallocating device memory using the CUDA API
• Transferring data between host and device with the CUDA API
• Launching kernels and synchronizing threads using the CUDA API
• Handling errors and exceptions with the CUDA API

CUDA C/C++

• The role of CUDA C/C++ in device programs
• Writing kernels in CUDA C/C++ to execute on the GPU and manipulate data
• Using CUDA C/C++ data types, qualifiers, operators, and expressions
• Leveraging built-in functions for math, atomic operations, and warps
• Utilizing built-in variables like threadIdx, blockIdx, and blockDim
• Using libraries such as cuBLAS, cuFFT, and cuRAND

CUDA Memory Model

• Distinguishing between host and device memory models
• Applying CUDA memory spaces: global, shared, constant, and local
• Working with CUDA memory objects: pointers, arrays, textures, and surfaces
• Managing memory access modes: read-only, write-only, read-write, etc.
• Understanding the CUDA memory consistency model and synchronization mechanisms

CUDA Execution Model

• Distinguishing between host and device execution models
• Defining parallelism using CUDA threads, blocks, and grids
• Using thread functions such as threadIdx, blockIdx, and blockDim
• Employing block functions like __syncthreads and __threadfence_block
• Utilizing grid functions, gridDim, gridSync, and cooperative groups

Debugging

• Identifying common errors and bugs in CUDA programs
• Inspecting variables, breakpoints, and call stacks using the Visual Studio Code debugger
• Debugging CUDA programs on Linux with CUDA-GDB
• Detecting memory errors and leaks using CUDA-MEMCHECK
• Debugging and analyzing CUDA programs on Windows using NVIDIA Nsight

Optimization

• Factors influencing CUDA program performance
• Improving memory throughput with CUDA coalescing techniques
• Reducing memory latency through CUDA caching and prefetching
• Optimizing memory accesses and bandwidth using shared and local memory
• Measuring and improving execution time and resource utilization with CUDA profiling tools