BLOG ILLUSTRATION / EXECUTION STORY

Update labels, train again, deploy again

This RECAP round occurs after pretraining and task SFT. It collects experience, updates advantage labels, trains the policy again and deploys it again. Performance evaluation is necessary to show an improvement.

Update labels, train again, deploy againRECAP starts this stage after offline pretraining and task SFT. It collects autonomous rollouts and corrections and expands data. It updates value-based advantage labels, trains the conditioned policy again and deploys it again. Label quality depends on evaluation quality. This illustrative round makes no claim of a measured gain. After pretraining + task SFT Current policy Advantage condition Robot rollout Outcome + corrections Expanded data Rollouts + corrections Relabel advantage New value estimates Train policy again Advantage-conditioned Deploy again
RECAP deployment expands data with robot rollouts and corrections. Updated value estimates give new advantage labels for the data. RECAP trains the advantage-conditioned policy again and deploys it again. Offline pretraining and task-specific SFT occur before this loop. The author made this explanatory diagram. The playback is an example, and animation time is not measured runtime. Open full-size SVG