F.01 · Transformer Core
Scaled Dot-Product Attention
Attention(Q, K, V) = softmax( QK⊤ ⁄ √dk ) · V
The core computation behind every LLM inference. Q·K⊤ scores are normalized by √dk to stabilize training, and FlashAttention with KV-cache cuts inference latency.
