BLOG ILLUSTRATION / EXECUTION STORY

Keep alignment of camera inputs with hand inputs

The visual branch uses channel concatenation of skeleton and video latents. The generator adds head and hand embeddings before causal output.

Keep alignment of camera inputs with hand inputsThe visual branch uses channel concatenation of skeleton and video latents. The generator adds head and hand embeddings before causal output. Head pose 6 DoF camera control Camera encoder Plücker-ray tokens Hand pose wrist + 20 angles/hand Motion encoder 3D hand tokens 2D skeleton image-aligned hand pose 3D VAE 2D skeleton tokens Token fusion + DiT next latent chunk Output frames
Author-redrawn Generated Reality control diagram. The visual branch uses channel concatenation of video and skeleton latents. Camera geometry and 3D hand articulation use different encoders. The generator adds their embeddings before chunk generation. New tracking supplies input at the next chunk boundary. This diagram does not show autoregressive-history internals, decoder detail or original result frames. Geometric control alone is not evidence of correct contact or object state. Open full-size SVG