SM89 / core baselineAda
Established core baseline for warp, memory, occupancy, and classic Tensor Core work.
Twelve interactive atlases connect visual GPU foundations, tile-level programming, Blackwell-aware optimization, and distributed inference in a 12-week evidence route that makes supported architecture and backend boundaries visible.
Learn theory, code, and measurement methods in GPU Kernel Atlas; explore GPU components and execution steps through interactive 3D scenes and a timeline in GEX. Then apply what you learned in the labs here.
GEX uses conceptual educational models; it does not present a physical chip floorplan, cycle-accurate simulation, or measured performance results.
GPU provides the compute foundation of the learning system. Follow these paths by topic; each application runs independently and your progress is not transferred.
Each architecture label describes concept applicability and maturity; it does not assume that you own the newest hardware.
SM89 / core baselineEstablished core baseline for warp, memory, occupancy, and classic Tensor Core work.
SM90 / currentCurrent foundation for TMA, warp-group MMA, distributed shared memory, and persistent scheduling.
SM100 · SM120 / currentSM100 and SM120 are distinct targets. Do not assume SM100 TMEM or cluster-aware paths apply to SM120; verify Tensor Core and FP4/FP8 support for the selected kernel.
SM107 / previewPreview material limited to capabilities documented by current first-party toolchains.
Preview content depends on supported toolchains.First-party sources are refreshed before publication; document freshness, feature maturity, and architecture/backend applicability are evaluated separately.
Core: hardware-generation-independent skills and evidence required for completion.
Current: supported by fresh first-party evidence; architecture and backend applicability are verified separately.
Preview: an exploration path dependent on toolchain or hardware maturity; it is not a graduation requirement.
Interactive laboratories are educational simulations; they do not claim measured hardware results.
A kernel is not complete without a reference, a shape/dtype matrix, and clean sanitizer results.
No performance claim without warm-up, quantiles, profiler evidence, and a controlled baseline.
The real goal is a portfolio-grade operator that works in PyTorch, compile, and serving workloads.
A route from visual foundations to capstone. Every atlas includes an interactive lab, a Concept Studio, a decision model, and an acceptance artifact.
Suggested pace: 14–16 hours per week. Explore concepts in the browser; produce working kernels, correctness tests, and measurement reports in your own GPU environment.
Record GPU/backend, compute capability, toolchain, and measurement context.
Grid, warp collaboration, divergence, and tile-level problem decomposition.
Coalescing, tensor descriptors, TMA/DSMEM applicability, and the TMEM boundary.
torch.library, triton_op/wrap_triton, opcheck, and structured autotune.
Grouped work assignment, routing, and profiler-backed operator selection.
FP8/MXFP8 scale metadata, accumulation, and quality guardrails.
Stable softmax, paged KV-cache, and FP4/FP8 applicability boundaries.
Reference, tolerance, alias/determinism, memcheck, racecheck, and TMEM guardrails.
Report merge, clustering, instruction mix, scoreboards, and CUDA Graph nodes.
C++/CuTe → PTX/SASS evidence, Tensor Cores, and architecture-gated persistent/grouped scheduling.
Encode/prefill/decode, graph boundaries, DP/TP/PP/EP, and topology evidence.
TTFT/ITL/throughput report, two 15%+ fusions, and defense
CUDA/Triton dual implementation
median gain across two fused kernels
Nsight studies
vLLM TTFT/ITL/ throughput report
interview defense
These thresholds are suggested capstone targets, not guaranteed speedups or results measured by this application. Compare against a baseline on the same hardware and workload; report the result and bottleneck when the target is not reached.