Series

The Life of a Token Across the LLM Stack

A six-part map of what happens between a user prompt and a completed answer: model computation, GPU execution, cache state, serving policy, full-stack diagnosis, and inference-time reasoning budgets.

Transformer computation GPU matmul kernels KV-cache memory Serving diagnosis Reasoning budgets
  1. Part 0

    LLM Inference Is a Full-Stack Systems Problem

    A full-stack map of LLM inference across Transformer computation, GPU kernels, KV-cache memory, request scheduling, and streamed token delivery.
  2. Part 1

    The Life of a Token Inside a Transformer

    How token IDs become contextual representations through embeddings, normalization, attention, MLPs, positional encoding, KV cache, and logits.
  3. Part 2

    Why Transformer Performance Is Mostly a Matmul Problem

    How Transformer linear layers become tiled GPU matrix multiplications shaped by memory movement, tensor-core utilization, batch shape, and arithmetic intensity.
  4. Part 3

    The Life of a Request Inside vLLM

    How vLLM-style serving engines coordinate prefill, decode, continuous batching, paged KV cache, scheduling, speculative decoding, and streaming.
  5. Part 4

    A Token's Journey Across the LLM Stack

    A unified diagnostic model connecting token computation, GPU kernels, KV-cache memory, request scheduling, latency, and throughput across LLM inference.
  6. Part 5

    Reasoning Effort Is an Inference-Time Systems Knob

    Reasoning effort changes hidden token use, latency, cost, context pressure, request residency, and serving capacity across the LLM inference stack.