BLOG ILLUSTRATION / EXECUTION STORY

Attention weights mix a separate value branch

Q and K give scaled scores. A causal mask and softmax give attention weights, which mix the separate V branch.

Attention weights mix a separate value branchQ and K give scaled scores. A causal mask and softmax give attention weights, which mix the separate V branch. Hidden X token vectors Q queries K keys V values Scores QK^T / sqrt(d_k) Mask + softmax causal weights Weighted sum weights × V
Q and K give scaled scores. Masking and softmax give attention weights from these scores. V supplies the values on a separate branch. The author made this explanatory diagram to show data dependencies, not a kernel launch schedule. The playback is an example. Animation time is not measured runtime. Open full-size SVG