AI Glossary
The complete dictionary of Artificial Intelligence
Thread Divergence
Phenomenon where threads within the same warp take different execution paths, causing branch serialization and significantly reducing parallel performance on GPU.
Shared Memory Bank Conflicts
Contention that occurs when multiple threads from the same warp attempt to simultaneously access the same memory bank of shared memory, causing access serialization.
Warp Scheduling
GPU scheduler mechanism that optimizes warp scheduling to maximize compute unit utilization and hide memory latency.
Register Spilling
Phenomenon where the compiler must move data from registers to local memory (slow) when registers are insufficient, significantly degrading performance.
Instruction Throughput
Measure of the number of instructions that can be executed per clock cycle, optimized by favoring native arithmetic operations and avoiding complex instructions.
Grid Stride Loop
Loop pattern where each thread processes multiple elements spaced by the total grid size, allowing processing of datasets larger than the thread grid.
Loop Unrolling
Optimization technique that eliminates loop iterations by duplicating the loop body, reducing loop control overhead and increasing instruction-level parallelism.
Memory Latency Hiding
Strategy of launching enough warps so the GPU can switch to ready warps while others wait for memory accesses.
Vector Memory Operations
Instructions that transfer multiple data simultaneously (float2, float4) between global memory and registers, improving effective bandwidth.
Cooperative Groups
CUDA API allowing flexible and collective synchronization between threads beyond traditional block boundaries, optimizing complex communication patterns.
Texture Memory Caching
Use of texture memory with its optimized cache for spatial accesses, particularly effective for access patterns with 2D locality.
Atomic Operations Optimization
Techniques to reduce contention on atomic operations, notably by using shared memory for local aggregation before global update.
Kernel Launch Overhead
Time cost associated with launching a GPU kernel, minimized by merging multiple small kernels into one larger one or using dynamic parallelism.
Work Distribution Balance
Optimization of work distribution between threads to avoid load imbalance where some threads finish much earlier than others.
Prefetching Strategy
Anticipatory technique of loading data into shared memory before their use, thus hiding the latency of global memory accesses.