CUDA — Compute Unified Device Architecture

Fact

NVIDIA’s toolset that lets programs use GPUs for many math jobs at once.

In Plain Words

CUDA gives your graphics card a work badge. It stops drawing dragons and shows up in a hard hat to do math.

It lets programs send huge math jobs to the GPU. It sits under AI training and fast inference.

Related Concepts

• GPU Compute
• PyTorch Deep Learning Framework
• LLM inference engine
• Flash Attention

GPU
CUDA turns the GPU from a graphics chip into a programmable math worker.

PyTorch
PyTorch often uses CUDA to call the GPU for deep learning work.

Inference engine
An inference engine often uses CUDA to run fast low-level math.

Flash Attention
Flash Attention is often built on CUDA for high speed.