BLOG ILLUSTRATION / EXECUTION STORY

Commit the accepted prefix, then add the correction

Examine the speculative-decoding rejection example. Examine the committed context, not only the highlighted tokens.

Speculative decoding keeps the prefixFour draft tokens d1, d2, d3 and d4 enter target verification with the existing context. The example acceptance outcome keeps d1 and d2, first rejects d3, and discards d4. That discarded candidate is not committed. Orange shows this rejection and the correction/resampling path to c3. Only d1, d2 and c3 extend the existing context. The diagram shows only the rejection branch, without probabilities or acceptance and correction sampling rules.Draft suffixcandidate tokensOutcomeexample onlyCommittedcontext +d1d2d3d4Target verificationd1acceptedd2acceptedd3first rejectd4discardd1d2c3Correction c3
The diagram shows only one rejection branch. It does not show acceptance probabilities, correction sampling or the all-accepted branch. Open full-size SVG