BLOG ILLUSTRATION / EXECUTION STORY

Search quantities in MuZero

h encodes observation history. Search queries f for policy and value, and g for candidate next state and reward. A selected action enters environment execution. The new observations then enter the history. A g expansion sends its reward to search and its next state to f.

Search quantities in MuZeroh encodes observation history. Search queries f for policy and value, and g for candidate next state and reward. A selected action enters environment execution. The new observations then enter the history. A g expansion sends its reward to search and its next state to f. Observation, encoding, search, action History Observations Encoder h Initial state Latent search g and f again Execute action Search selects it. Dynamics g Next state + reward Prediction f Policy prior + value Action Reward Next state New observation
MuZero uses h for history encoding, g for latent transitions and rewards, and f for policy/value predictions in search. After environment actions, h encodes new observations. Author-redrawn mechanism from Schrittwieser et al. The source figure keeps the original board snapshots, full search tree and training-unroll panel. Source: Schrittwieser et al. Playback time is illustrative and is not measured runtime. Open full-size SVG