Learn the frontier · Recreations

Recreations

Built from scratch, in runnable code you can keep.

First-party paper rebuilds as minimal, annotated, runnable code. Warm narrative build logs with our own repo, honest scope, and how our numbers came out — a guided tour you can fork, run, and learn the technique from the inside out.

RSS ↗1 piece

CHECKPOINT 00052026-05-28scope: minimal

Recreating FlashAttention: A Tiled, IO-Aware Attention Kernel from Scratch

The minimal Triton kernel recovers FlashAttention's headline behavior — O(N) HBM traffic and bit-exact outputs against a reference — so the win we reproduce is memory bandwidth, not FLOPs; matching the CUDA kernel's absolute speedups, though, is out of reach at this scope.

flash-attention kernels attention gpu-memory