{
  "title": "One chat request across engine processes",
  "caption": "Author-created trace from vLLM v0.32.0 and SGLang v0.5.23 with default settings and one GPU. With tensor parallelism above 1, both engines add worker or scheduler processes per GPU. Playback time is not measured latency.",
  "alt": "Flow diagram: a chat request goes from the frontend to the scheduler and GPU worker, then returns through the frontend or a separate detokenizer",
  "sources": [
    {
      "title": "Original author diagram",
      "file": "diagram.svg",
      "sha256": "fb45e23501155242baf987c28329e5bc572df518fb784ace09f30d5f4fbfc8fa"
    },
    {
      "title": "vLLM v0.32.0 frontend client sockets",
      "url": "https://github.com/vllm-project/vllm/blob/v0.32.0/vllm/v1/engine/core_client.py#L659-L669"
    },
    {
      "title": "vLLM v0.32.0 output processor",
      "url": "https://github.com/vllm-project/vllm/blob/v0.32.0/vllm/v1/engine/output_processor.py#L765-L775"
    },
    {
      "title": "vLLM v0.32.0 single-process executor",
      "url": "https://github.com/vllm-project/vllm/blob/v0.32.0/vllm/v1/executor/uniproc_executor.py#L51-L69"
    },
    {
      "title": "SGLang v0.5.23 process layout",
      "url": "https://github.com/sgl-project/sglang/blob/v0.5.23/python/sglang/srt/entrypoints/engine.py#L248-L260"
    },
    {
      "title": "SGLang v0.5.23 stop-string check",
      "url": "https://github.com/sgl-project/sglang/blob/v0.5.23/python/sglang/srt/managers/schedule_batch.py#L1910-L1925"
    },
    {
      "title": "Article evidence and version ledger",
      "file": "../../illustrations/SOURCE.md"
    }
  ],
  "limitations": [
    "The traces assume default serve settings, one node, one GPU and a streaming chat request.",
    "With tensor parallelism above 1, vLLM adds one worker process per GPU and SGLang adds one scheduler process per TP rank.",
    "Steps are a teaching order. Playback duration does not measure latency."
  ]
}