A streaming video model must continue to emit frames. It must keep its generated history and accept new controls during generation. Sampling, memory and training must be compatible with that ongoing generation.

Keep the world coherent while frames stream

Generate a short clip. Supply its last frames as input for the next clip. Tell the model to generate one more clip. This is sufficient for demos and bounded extensions. Stable streaming is less easy. It must keep the world coherent when its own predictions become new input.

A car drifts a small number of pixels, and the next chunk receives that drift as input. A face becomes overexposed. Subsequent frames may use that appearance. An object disappears behind a wall and returns with a different shape. The system must keep identity, geometry, motion, lighting, layout and the state that determines what occurs next.

Looping, fading, slowing down or hiding motion can all make a sequence longer. A response to new controls is also necessary during streaming. The future must stay connected to the past.

These requirements are necessary for the running generator:

• The model can emit completed frames before it completes all future frames.
• The model can keep past state that helps generation without repeated processing of all past frames.
• The model can receive new control signals during generation.
• The sampling cost must let the model give output with low latency.
• Training supplies the model with samples from its inference-time distribution.

Causal generation predicts the next output from the history available at that point. Autoregressive generation uses each output as input for subsequent steps. These two choices and retained memory make streaming possible. Offline stages and short-window refinement can still help. The central question is the state kept between chunks, not only the video duration.

The state kept between chunks

A camera leaves a room, turns through a corridor and returns. The generator must recover the room from its earlier frames and follow the new camera motion.

This article uses this generation/update pair to make groups of papers. The pair gives their requirements:

Here $\hat z$ is a generated frame/chunk represented by compressed values, or latents. $m_t$ is accessible history. $a_t$ is an optional action, and $c_t$ is text, camera, pose or other controls. $U_\theta$ updates history with cache insertion/eviction, compression, retrieval or recurrent-state updates. A key/value (KV) cache keeps attention keys and values for reuse.

Neither latents nor a KV cache necessarily captures all relevant physical state.

Video time and denoising steps use two different indices. Video time $t$ advances when chunks become history. Denoising steps remove noise in one chunk. A teacher supplies training supervision. It may have temporal context unavailable to the deployed student. Each method changes a different part of this setup.

Five problems in one streaming loop

This article divides the remaining problems into five layers. The questions are about output availability, history used in training, teacher inputs, memory and control tests. Sampling with a small number of denoising steps sets the response speed for all five.

Self-Forcing changes the histories used in training. MilliVid changes their representation. Causal Forcing changes supervision. The methods are for different failures. A causal model may forget a room. A model with rich memory may ignore a new action.

Why extending short clips reaches a limit

The earlier long-video literature did not start with native streaming. A 2024 survey on long video generation gives two broad groups of methods. Divide-and-conquer systems divide a long video into manageable parts. Temporal autoregressive systems extend generation step by step.

The difference is important for serving. Divide-and-conquer methods may stay offline during scene stitching, motion interpolation, clip extension or short-generation planning. Temporal autoregression is closer to streaming, but may still only generate blocks and concatenate them.

The chunk-wise survey gives the reason for this choice. Denoising a full tensor with a fixed length and resolution can exhaust memory if the length or resolution increases. Short chunks make each subproblem smaller. Conditioning across chunks connects them.

Chunking keeps the short model’s limits: a small active window, a growing or truncated cache and local errors reused as history. Stable streaming must select the state to keep from one chunk to the next.

Rules for frame finalization

In conventional fixed-window video diffusion, early and late frames supply information to one another through bidirectional attention. Repeated denoising operates on the full window again. This helps a closed clip but causes four problems online:

• A rule for frame finalization is necessary before frame emission.
• Repeated denoising of a growing window costs more.
• Denoising with past and future together makes KV-cache reuse less easy.
• A new camera turn, prompt, pose or robot action should change future frames without changes to the emitted clip.

As with a language model, history that the generator can cache, advance and reuse is necessary. Video makes this history larger and less discrete. An incorrect latent can change motion, geometry, lighting or identity.

What can be emitted while the future is uncertain?

Diffusion Forcing assigns different noise levels to tokens or frames, rather than one level to a full sequence. In shorthand:

Here $z_i$ is the latent at position $i$, $\tau_i$ is its noise level and $\epsilon_i$ is noise. One sequence can now contain clean past, partly denoised present and noisy future.

A diagonal denoising frontier moves through video time. Frames behind it are closer to final. Frames ahead of it stay uncertain. This lets a streaming generator keep completed history and work on the near future. Subsequent frames can stay flexible.

Diffusion Forcing puts stable history and uncertain future in the same sequence. Training on histories that generation will produce is a different problem. Memory selection is also a different problem.

Why training on real history leaves a rollout gap

Generated chunks return as input to subsequent predictions. The second layer asks if training supplies the model with these histories.

During teacher-forced training, the model has real history as input:

During inference, it has generated history as input:

This train/test mismatch is exposure bias. Continuous video errors can accumulate. Color shifts become overexposure, geometry errors become new wall boundaries, and identity drift changes the face used as subsequent input.

Self Forcing trains the student through autoregressive rollouts with its own generated past and rolling KV cache. A teacher still supplies supervision for the generated distribution. The student learns from the kind of context that it will make at inference.

The student has its own failures during training, and teacher supervision and gradients can correct them. Full backpropagation through long rollouts is expensive. Thus, the method truncates or samples gradient propagation. This keeps self-generated context in training but limits how far learning reaches.

The ablations change training history and distribution-matching objectives. The argument is about generated context, beyond any single loss. The paper reports degradation beyond training length and limited long-range learning with gradient truncation. How far into generated history does correction reach?

Training on subsequent failures

A model trained on its own short rollouts may still fail after a minute. Training must reach the states where those subsequent failures appear.

Self-Forcing++ runs the student through a long trajectory, then samples short windows from subsequent points. A short-horizon teacher corrects their local distribution toward plausible video. The teacher supplies supervision for these windows without generation of the full long sequence.

The student runs the long trajectory, and the teacher corrects short windows. The paper reports videos up to 4 minutes and 15 seconds. I think its broader lesson is to train on late failures before they occur in a demo.

The official project page reports a remaining lack of long-term memory. Occluded objects can change after long absences. Continuous drift can change visible objects in very long rollouts. Thus, training at a subsequent horizon does not settle what persistent state must store.

Record video duration. Also ask how far into a self-generated rollout training supplied correction.

Do teacher and student have the same conditional problem?

Distillation trains a student with supervision from another model. A strong bidirectional teacher can give good clips but stay slow and non-causal. Streaming makes an incremental student, a reusable KV cache and a small number of sampling steps necessary. Agreement between histories alone does not answer this supervision question.

CausVid, formally From Slow Bidirectional to Fast Autoregressive Video Diffusion Models, distills a bidirectional teacher into an autoregressive generator. It reports a 50-step teacher, a 4-step generator and a single-GPU frames-per-second (FPS) figure. I would compare the supervision design before that figure. These papers use different resolutions, hardware, sizes and latency definitions.

A bidirectional teacher has future frames unavailable to a causal student. Causal Forcing finds a specific problem in ordinary differential equation (ODE) initialization. With future context hidden, a noisy frame can be compatible with several clean endpoints. Regression on the teacher’s incompatible targets can average them.

The method trains an autoregressive teacher, distills causal ODE trajectories, then uses asymmetric distribution matching. That last stage may still use a bidirectional teacher. The repair is to the supervision path.

Context Forcing is for a teacher with too little history. A short-window teacher cannot correct dependencies across a long student rollout. The method uses a long-context teacher and slow-fast memory to manage growth.

Find the teacher’s available past and future inputs. Do a check of the supervision’s compatibility with the student’s one-way generation.

What should survive a bounded memory budget?

A corrected rollout can also forget a room if its retained history loses necessary information:

What should hundreds of frames of history look like inside the model?

The naive representation is uniform. If a latent video has $T$ frames and each frame uses an $H \times W$ token grid, context cost scales as:

Spatial detail is necessary in recent frames for motion, appearance and continuity. Layout, camera path, object identity, revisitation and rough geometry may be the main requirements for distant frames. Full spatial resolution for a frame from twenty seconds ago is frequently wasteful.

MilliVid trains a hierarchical autoencoder with independently decodable latent levels. Fine levels keep spatial detail. Coarse levels use a very small number of tokens for each frame. Generation first makes a long coarse plan, then refines the relevant time region.

The paper gives an example: 30 seconds at 20 fps, with a 16 by 16 latent grid for each frame. The result is 600 x 16 x 16 = 153,600 tokens. Attention to all of them may cost too much or bury relevant history in fine detail. Storage of more frames alone cannot ensure that the model uses them.

MilliVid’s LOOPCRAFT uses long Minecraft-style sequences with actions and pose metadata for tests of world preservation and revisitation. Local plausibility can coexist with a forgotten camera origin. Global consistency can coexist with imperfect texture. The tests must distinguish these differences.

Frame packing changes the number of frames that memory can hold. FramePack, a context-packing method for next-frame video models, is the baseline here. It compresses older frames into fewer tokens. Independently decodable coarse levels change how much detail is necessary for each old frame. This is a different choice.

A newly trained hierarchical autoencoder is necessary for MilliVid. Thus, it is less easy to add MilliVid to fixed-latent WAN- or Hunyuan-style models. The paper also reports extra rollout steps compared with simpler packing baselines. Its representation is for a different problem from Self-Forcing.

Self-Forcing asks: does the model train on its own generated context?

Self-Forcing++ asks: does training reach the horizon where failures appear?

MilliVid asks: does each part of long-range history keep the right level of detail?

The analysis finds that hierarchical context alone does not reproduce MilliVid’s gains in FramePack. Farther-ahead coarse targets give better consistency at some horizons despite less sampling context. The authors hypothesize that these targets teach the model to use distant history. I think this is a reason to design memory and prediction tasks together.

Memory selection and storage

KV caching reuses past attention keys and values. In streaming video, retention also selects the generated visual history that stays accessible.

The memory question has at least four parts:

Memory selection: Which historical tokens should stay available?

Memory compression: How many bytes can be used for each historical token?

Memory retrieval: Can the model find relevant non-local history, not only the recent window?

Level of detail: At what spatial and semantic resolution should history be stored?

A sliding window keeps recent history. Attention sinks keep a small number of early tokens permanently for stable attention. Slow-fast memory keeps recent dense history and slower persistent state in different parts. Sparse policies keep selected blocks.

Quant VideoGen uses 2-bit KV quantization to decrease the bytes necessary for history, not to propose a new world model. Future Forcing selects entries by estimated future-query requirements. LongLive-RAG makes previously generated latents searchable beyond the recent window.

VideoSSM uses a local window and evolving global state-space memory together. That recurrent summary moves toward persistent scene state. Broad interactive control is still to be tested.

A future system could use retrieval of distant episodes, quantized storage, hierarchical detail and recurrent or object-level state outside the 2D latent grid together. A token cache alone does not automatically identify objects behind walls, keep occluded identities or map a revisited room to its camera pose.

Sampling and the latency budget

Dozens of denoising steps for each chunk can make a causal model too slow. Sampling cost is a limit on state updates, self-generated training rollouts, long memory and control responsiveness across all five layers.

Distribution-matching, consistency and diagonal distillation, with adversarial losses, target this cost. Consistency distillation trains a student to go from a noisy input to a clean output in one or a small number of steps. Diagonal Distillation uses more denoising work early and decreases the steps for subsequent chunks. One-Forcing uses a distribution-matching distillation (DMD) objective and an auxiliary generative adversarial network (GAN) loss together for stable one-step generation. Causal Forcing++ targets one or two steps for each frame. Ultra Flash is for high-resolution real-time generation.

CausVid, Self Forcing, Diagonal Distillation, Causal Forcing++, One-Forcing and Ultra Flash report speeds in different conditions. Compare resolution, hardware, model size, first/inter-frame latency, steps and output frame-rate conventions before a comparison of their numbers.

A streaming claim makes quality and consistency necessary measurements, with denoising steps, time to first frame, inter-frame latency and hardware.

Can new controls change the future without a world reset?

In the fifth layer, $a_t$ and $c_t$ can change while $m_t$ carries the world those controls act on. The model must follow the new condition and keep that world.

This connects streaming to interactive world models and robot control. Long videos can capture visual patterns but still fail object permanence, contact physics or reliable causal prediction. Duration alone cannot show physical correctness.

A prompt edit, camera turn, robot action or tracked human pose may arrive during generation. The generator must use it without a world reset.

VRAG uses action conditioning and retrieved global state to address compounding errors and insufficient memory. It also shows the limits of naive long context or retrieval. CausalCine handles multi-shot narratives, dynamic prompts and content-based memory routing. Directing the World uses human-motion and camera-trajectory control together in fast autoregressive generation.

These are different benchmarks. Together they give a reason to do tests of a generator’s compliance with a new condition and its preservation of state. The state includes identity, geometry and usable memory. Acceptable user-visible latency is also part of this test.

For robot control, the important prediction keeps what is necessary for action. Streaming research can help identify which visual state must persist. It can also help identify what can be forgotten and how to correct errors before a policy uses them.

Unsolved problems

Longer context helps when the model can represent, find and use relevant history in its time and memory limits.

First, retrieve history by what it contains. When the camera turns back, a room visited thirty seconds ago may matter more than the last four frames. LongLive-RAG starts toward this goal. Better retrieval keys are still necessary for object identity, layout, pose, action state and intent.

Second, use representations together where tasks make this necessary. Token caches, hierarchical latents and recurrent summaries each keep different information. None alone guarantees object/scene state for occlusion, possible actions (affordances), contact and revisitation. A hybrid could use local tokens, hierarchical long history, retrieved episodes and explicit objects or geometry.

Third, new controls can destabilize a rollout. If the control simply overwrites the existing world state, the video may follow the instruction but lose identity or geometry. If the existing world state dominates, the model ignores the control and the interface feels unresponsive. Measurements of compliance and continuity are necessary in the test.

Fourth, Fréchet Video Distance (FVD) and short-clip preference metrics leave room revisitation after thirty seconds untested. Repetitive loops can give a false appearance of stable motion. Give first-frame time, inter-frame latency, throughput, resolution, steps and hardware separately, rather than one FPS number.

Finally, do a test of the combination. Causal rollout defines what can be emitted. Self-Forcing changes training histories, and Causal Forcing aligns supervision. MilliVid, VideoSSM, Quant VideoGen, Future Forcing and LongLive-RAG change what memory keeps and how the model accesses it. Action-conditioned methods accept controls. Causal Forcing++, One-Forcing, Diagonal Distillation and Ultra Flash target sampling cost.

Tests of a combined system itself are necessary. Results from different papers cannot validate that system.

The next test I would do

I would judge the next streaming models by whether the next frame belongs to the same world, rather than by memory of every pixel.

I propose this first test:

1. Move the camera out of a room.
2. Continue generation for thirty seconds.
3. Turn back with a new control, such as a camera path or prompt edit.
4. Give a score for the room’s layout and object-identity continuity.
5. Give a score for the model’s compliance with the new control.
6. Give time to first frame, inter-frame latency, denoising steps and hardware with those scores.

My synthesis is a five-layer serving design: causal emission, generated-history training, compatible teacher supervision, persistent memory and tested online control. Sampling with a small number of steps must make these choices sufficiently responsive for operation.

Operation of such a model resembles operation of a fragile simulator with visual, latent, cached and controlled state. Generation of another frame is only the start. The model has to continue the same world.

@article{ji2026streamingautoregressivevideogeneration,
  title = {Streaming Autoregressive Video Generation},
  author = {Ji, Wenbo},
  journal = {fusheng-ji.github.io},
  year = {2026},
  month = {June},
  url = {https://fusheng-ji.github.io/blog/posts/streaming-autoregressive-video-generation/}
}

References

• A Survey on Long Video Generation: Challenges, Methods, and Prospects, Chengxuan Li et al., 2024.
• Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion, Boyuan Chen et al., 2024.
• Towards Chunk-Wise Generation for Long Videos, Siyang Zhang and Ser-Nam Lim, 2024.
• From Slow Bidirectional to Fast Autoregressive Video Diffusion Models, Tianwei Yin et al., 2024.
• VRAG: Learning World Models for Interactive Video Generation, Taiye Chen et al., 2025.
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion, Tianwei Yin et al., 2025.
• Self-Forcing++: Towards Minute-Scale High-Quality Video Generation, Justin Cui et al., 2025. Official project page with algorithm and limitations.
• VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory, Yifei Yu et al., 2025.
• Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation, Hongzhou Zhu et al., 2026. Project page.
• Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization, Haocheng Xi et al., 2026.
• Context Forcing: Consistent Autoregressive Video Generation with Long Context, Shuo Chen et al., 2026.
• Streaming Autoregressive Video Generation via Diagonal Distillation, Jinxiu Liu et al., 2026.
• CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives, Yihao Meng et al., 2026.
• Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation, Min Zhao et al., 2026.
• One-Forcing: Towards Stable One-Step Autoregressive Video Generation, Jiaqi Feng et al., 2026.
• Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation, Jiayi Luo et al., 2026.
• LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation, Qixin Hu et al., 2026.
• MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation, Ishaan Preetam Chandratreya et al., 2026.
• Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions, Jie Huang et al., 2026.
• Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control, Haoyuan Wang et al., 2026.

Updated 08 November 2026