Tokenized state enters scheduling, KV allocation/reuse and a selected model batch. Sampling gives new IDs for streaming. Then the engine updates request state. The scheduler can schedule an unfinished request again. When a request is completed, it releases its references. Shared or reusable physical blocks may stay.
The engine does scheduling, KV management, execution, sampling and state updates again until the request is completed. The constructed example continues one time, then stops. Completion releases request references. It does not necessarily free shared or reusable physical blocks. The author made this explanatory diagram. Open full-size SVG