{
  "title": "One RL step hands weights to the rollout engine",
  "caption": "Author-created trace from verl v0.9.1 (HYBRID mode, vLLM rollout) and AReaL v2.1.0 (xccl weight updates). Function names follow the pinned source. Playback time is not measured latency or memory use.",
  "alt": "Flow diagram: an RL controller sends prompts to a rollout engine, the trainer learns from buffered trajectories, and new weights return to the engine",
  "sources": [
    {
      "title": "Original author diagram",
      "file": "diagram.svg",
      "sha256": "69ea731a4a29f092d19a6ac269f4a1374c2dacbf6b9f2bb52d12441c09072e1e"
    },
    {
      "title": "verl v0.9.1 synchronous trainer",
      "url": "https://github.com/verl-project/verl/blob/v0.9.1/verl/trainer/ppo/v1/trainer_sync.py"
    },
    {
      "title": "verl v0.9.1 colocated weight update",
      "url": "https://github.com/verl-project/verl/blob/v0.9.1/verl/workers/engine_workers.py"
    },
    {
      "title": "verl v0.9.1 vLLM rollout server",
      "url": "https://github.com/verl-project/verl/blob/v0.9.1/verl/workers/rollout/vllm_rollout/vllm_async_server.py"
    },
    {
      "title": "AReaL v2.1.0 staleness manager",
      "url": "https://github.com/areal-project/AReaL/blob/v2.1.0/areal/infra/staleness_manager.py"
    },
    {
      "title": "AReaL v2.1.0 remote inference engine",
      "url": "https://github.com/areal-project/AReaL/blob/v2.1.0/areal/infra/remote_inf_engine.py"
    },
    {
      "title": "Article evidence and version ledger",
      "file": "../../illustrations/SOURCE.md"
    }
  ],
  "limitations": [
    "Each trace follows one configuration at the pinned tag. verl and AReaL also support other placements and weight transports.",
    "The frameworks pin older engine releases than vLLM v0.32.0 and SGLang v0.5.23. The trace describes the framework code, not the newest engine API.",
    "Steps are a teaching order. Playback duration does not measure latency, memory use or throughput."
  ]
}