BLOG ILLUSTRATION / EXECUTION STORY

Initialize from VLM weights and train on robot data

The training recipe uses visual-language foundation weights and the two robot-data sources to train a generalist policy.

Initialize from VLM weights and train on robot dataInternet-scale vision-language pretraining supplies foundation weights. Robot training uses Open X-Embodiment and PI robot data to give a generalist policy. The arrows show a training recipe. They do not show a layer-exact optimizer trace or robot execution results. Internet pretraining Vision + language VLM foundation Initial weights Open X-Embodiment PI robot data eight configurations Robot training π0 policy parameters Generalist policy Use prompts or fine-tune
The author made this diagram from Physical Intelligence’s π0 source figure. Internet-scale vision-language pretraining supplies the backbone. Robot training uses Open X-Embodiment and PI robot data. After this training, the policy can receive prompts or be fine-tuned for tasks. The diagram does not show robot pictures or task icons, and it is not a layer-exact computation graph. The playback is an example, and animation time is not measured runtime. Open full-size SVG