An articulated robot arm moves an orange mug toward a plate on a table watched by a small camera. — Semantic context has to become continuous, executable action that can respond to observation.

From π0 to π0.7: A Tutorial on Open-pi and Robot Foundation Models

Wenbo Ji 25 May 2026 33 min read

In brief

A vision-language-action (VLA) policy uses visual and language context to generate robot commands. π0 generates continuous action chunks. Other open-pi work adds training signals, context and execution methods. Separate transfer, control and deployment tests are necessary for each mechanism.

π0 generates continuous action chunks. FAST compresses trajectories for token prediction. These representations have different training and control functions.
Use broad visual-language knowledge again. Adapt the state, action coordinates, cameras and data for each robot.
Subtasks, memory and optional visual goals are conditions for policies that support them. Real-time chunking is for inference delay. Local online RL is for precise contact.

Dated mechanism map: π0, FAST, π0.5, π*0.6, MEM/RLT and π0.7 have different training, inference and execution functions. — Fig. 1. The map gives release dates of the open-pi works and groups them by training, inference or execution.

The instruction “Put the mug next to the plate” gives a robot two different problems.

A vision-language model (VLM) can supply object and language knowledge. It can recognize the mug, interpret “next to,” locate the plate on the table and keep the instruction while the scene changes.

Smooth, continuous, high-frequency control is also necessary for the arm. Cameras supply images. Objects can slip, and inference uses time. A chatbot-style sequence of motor tokens is poorly suited to that immediate control loop.

Physical Intelligence’s work from π0 in October 2024 to π0.7 in April 2026 examines these two functions. I read the releases together to compare their different robot-learning and control functions.

I use the term “open-pi sequence” for this set of works. It includes model releases, infrastructure and research techniques for the π family.

My question is:

Which representation helps the model learn, select an action or execute it?

I use three decision types. The decisions can change each other. A change to one can occur without a change to the others:

Decision	What changes	Question
Training signals	Data, targets and gradients for policy training	What should the model learn? Which knowledge should a loss keep?
Inference conditions	Information available when a trained policy selects an action	What resolves ambiguity in the task, scene, history or desired outcome?
Execution constraints	Physical timing and contact requirements for a proposed trajectory	Which actions are committed, and which future actions can feedback change?

I write for readers who know transformers, VLMs, fine-tuning and reinforcement learning (RL).

A vision-language-action model (VLA) uses visual-language context to generate robot actions.

π0 and FAST use different action representations for training. Knowledge Insulation (KI) controls which losses update the VLM. Its purpose is to keep VLM knowledge during robot training. π0.5 and Human2Robot change the training data. RECAP, the training method for π*0.6, learns from deployment outcomes.

Hi-Robot, MEM (Multi-Scale Embodied Memory) and π0.7 supply subtasks, history and desired futures during inference. Hi-Robot has a high-level planner for a VLA. Real-time chunking (RTC) keeps a new action chunk consistent with motion during inference. RLT adds a small online RL learner for precise contact.

Methods can change more than one decision type. π0.5 uses mixed training data and subtask-conditioned inference. RECAP learns from execution outcomes and supplies advantage as a condition. I use these categories to examine the relations.

I also read external explanations. Hugging Face's π0/π0-FAST tutorial gives information about VLA and action representations. Cloderic's notes on π0 give architecture and data-recipe information. Roboflow's VLA overview gives more "VLM plus action" context.

Part 1: Training signals and action representations

First, the policy must learn motion and keep semantic knowledge. A continuous action distribution, a compact token target and a protected gradient path have different functions.

π0: Use visual-language knowledge again and generate continuous motion

π0, announced October 31, 2024, uses a pretrained VLM’s object, language, spatial and situational knowledge again. It does not learn all that knowledge again from robot data.

That VLM is not a high-frequency controller. Long discretized motor sequences strain a language-oriented output format. A different system must move the arm if the VLM supplies only high-level plans.

π0 keeps the VLM backbone and adds a continuous action expert.

The instruction is “Pick up the orange bowl”. The robot must recognize the object and interpret the instruction. Timing, grip closure and correction near contact are also necessary for the reach. That smooth trajectory is a different representation from the sentence.

π0 receives camera observations, a language instruction and robot state. The VLM supplies visual-language features to an added flow-matching action expert. Training updates the VLM and the action expert together as one model. They are not a separate planner and controller.

An action chunk contains the commands predicted together for a short horizon:

A_t = [a_t, a_{t+1}, \ldots, a_{t+H-1}]

Here $a_t$ is the command at control step $t$ . $H$ is the number of future commands that one call predicts. π-family examples frequently use a 50-action chunk, roughly one second. But the horizon is a design selection. Trajectory prediction amortizes a VLM-sized inference call over several high-frequency control steps. The controller moves during those steps.

The model learns a conditional distribution over full chunks, $p_\theta(A_t\mid o_t,l,q_t)$ . It does not learn only one next motor value. Here $o_t$ denotes camera observations. $q_t$ is robot state. The π0 paper, Section IV, gives generation with a learned velocity field.

In this tutorial notation, $s$ goes from noise to data. Mix a demonstrated chunk $A$ with Gaussian noise $\epsilon$ :

A^{(s)}=(1-s)\epsilon+sA,\qquad v_\theta(A^{(s)},s\mid o_t,l,q_t)\approx A-\epsilon.

Training gives the field the direction $A-\epsilon$ along these sampled paths.

At inference, start from noise. Integrate the learned field to get a chunk.

Different starts can give different plausible motions. A single average of those motions is not necessary.

Generation index $s$ is different from robot time $t$ . Integration changes the full chunk before the robot moves. Part 3 examines this difference when the robot executes part of a chunk.

Fig. 2. Visual-language representations condition a flow-matching action expert, and the policy’s motor interface receives robot state. The expert generates a full continuous action chunk. Generation index s is different from robot execution time t. Full interactive figure

The official π0 post gives a roughly 3B-parameter pretrained VLM as the starting point. The policy uses continuous action chunks to supply motor commands at up to 50 Hz. Its training mixture includes internet-scale visual-language pretraining, open robot data and Physical Intelligence's multi-robot dexterous dataset. The internal robot data has eight robot configurations. Tasks include bussing tables, folding laundry, assembling boxes, packing groceries and handling cables.

Fig. 3. Author-redrawn view of the π0 recipe. Internet-scale vision-language pretraining supplies the backbone. Open X-Embodiment and PI robot data support robot training, and the policy can use prompts or task fine-tuning. Source: Physical Intelligence π0 blog. Full interactive figure

π0 uses broad knowledge again and generates continuous control. But a chunk planned from an old camera frame can be stale when the controller executes it. I examine that timing problem in Part 3.

FAST: Compress motion before action-token prediction

FAST, published January 16, 2025, keeps autoregressive action prediction but makes its tokens shorter. Simple per-dimension binning uses many tokens that are not necessary for high-frequency dexterous motion.

An example is an arm that moves in the direction of a cup. In a short window, coordinates usually change smoothly. Thus, consecutive commands contain redundant information.

Naive tokenization bins each dimension separately. A seven-dimensional action at 20 Hz for one second can use 140 tokens. Higher frequency or longer horizons make that list longer. Prediction then uses steps for changes that compression can decrease.

FAST uses the action chunk as a compressible time series. Its pipeline is:

Normalize each action dimension with robust dataset quantiles.
Apply a Discrete Cosine Transform (DCT) along time.
Quantize the frequency coefficients.
Flatten the coefficients in low-frequency-first order.
Apply byte-pair encoding (BPE) to make common coefficient patterns into compact action tokens.

The DCT represents the low-frequency structure of smooth movement. It represents the broad reach before finer corrections, instead of each coordinate at each step.

The FAST report from Physical Intelligence gives typical chunks of 30 to 60 FAST tokens. It gives roughly a 10x compression compared with prior action tokenizations. The report gives a training-speed comparison: π0-FAST trains up to five times faster than the original π0. Its dexterity approaches that of the original π0.

FAST+ is a broader tokenizer trained on roughly one million real robot action sequences from multiple robot types.

FAST has two functions here. π0-FAST is a fully autoregressive VLA. It learns chunks with next-token prediction. Knowledge Insulation uses FAST action targets to train the backbone. A continuous expert supplies fast execution.

Fig. 4. FAST encodes demonstrated action chunks as compact frequency-structured tokens, but flow matching generates continuous chunks from noise. A data link between the two lanes is not necessary. In Knowledge Insulation, the token objective can train the backbone, and the continuous expert supplies inference-time actions. Full interactive figure

Compression makes action learning better, but physical timing is also necessary for control. FAST alone does not make a deployed robot responsive.

Knowledge Insulation: Adapt the backbone with separate losses

Adaptation of a VLM to robot actions can cause damage to its valuable semantic knowledge.

The publication date of VLAs that Train Fast, Run Fast, and Generalize Better is May 28, 2025. The work uses the name “motor cortex” for a new action expert. Its learning signal can change the pretrained representations in an incorrect direction.

End-to-end continuous action loss can favor the fit to robot demonstrations instead of object knowledge or instruction following. The official examples show that naive adaptation selects an incorrect object for the request.

Knowledge Insulation (KI) has different losses that update each set of parameters:

The VLM backbone uses language/web objectives and FAST-tokenized robot-action prediction for adaptation.
The continuous action expert attends to the backbone and learns flow-matching control.
The model stops gradients from the continuous action expert before they go into the VLM backbone.
The model discards FAST tokens at inference. The action expert supplies the actual continuous command.

A frozen backbone would keep its semantics but miss robot-relevant features. KI lets the backbone learn robot behavior from language/web objectives and FAST tokens. The continuous control loss does not reach the backbone. The cost is a more complex architecture. I think that cost is justified when broad knowledge and motor skill are necessary for a generalist policy.

π0.5: Broaden the training data for new environments

π0.5, published on April 22, 2025, is for open-world generalization. This includes long-horizon cleaning and rearrangement in homes and object configurations outside the robot training set. Its training mixture has more types of data, and it predicts high-level subtasks. This section examines the mixture. Part 2 examines the subtask interface.

The objective of π0.5 training is coverage of environments, bodies, high-level task labels and web-scale multimodal knowledge. The design does not assume that robot-data volume alone guarantees generalization.

The mechanism is heterogeneous co-training. The official report gives complementary signals in the training mixture:

Data signal	Role in π0.5
Mobile manipulator data (MM)	Long-horizon household behaviors in mobile settings
Multiple-environment non-mobile data (ME)	Physical skills in different scenes
Cross-embodiment data (CE)	Transfer from multiple robot morphologies
High-level subtask / verbal instruction data (HL)	Task decomposition and interactive language following
Web multimodal data (WD)	Object/category knowledge, visual question answering (VQA), grounding, captioning and detection

Each source has a different function. Mobile data adds long household behavior. Different environments decrease dependence on one room. Cross-embodiment examples help behavior depend less on one body. High-level labels teach decomposition. Web data increases object/category knowledge.

Co-training map: different robot-action data and language/web signals supply complementary learning signals to π0.5. — Fig. 5. Author-redrawn training view of π0.5. Mobile, multi-environment and cross-embodiment action data train physical behavior. Subtask/instruction and web data supply complementary context. The diagram shows heterogeneous co-training. Source: Physical Intelligence π0.5 blog.

The official ablations compare the data sources. In the release’s reported out-of-distribution (OOD) evaluation, full π0.5 has a 94% follow rate and a 94% success rate. Without web multimodal data, those values are 80% and 74%. Without cross-embodiment data, they are 67% and 49%. Without multiple-environment robot data, they are 33% and 31%.

I read these as complementary contributions in that evaluation. Web data helps unfamiliar-object knowledge. Different bodies and environments decrease dependence on one robot or room.

Human2Robot: Transfer after broad robot pretraining

Human demonstrations are abundant compared with costly robot data. But hands, viewpoints, timing, kinematics and commands are different. A human video contains no robot joint commands.

Emergence of Human to Robot Transfer in VLAs, published on December 16, 2025, examines broad robot pretraining. Can a π0.5 with sufficiently broad pretraining align human and robot behavior well enough to learn from these two sources?

The human data contains egocentric demonstrations with task/subtask structure. Training uses these demonstrations and robot demonstrations together. The human data is not casual internet video. The official release attributes transfer to broad VLA pretraining without bespoke alignment for each new behavior.

Transfer is conditional. After narrow robot pretraining, human actions can be too different in action space, view, timing or contact. Broader robot pretraining can help. Target-task and embodiment tests are also necessary.

The official project page gives a roughly 2x improvement over four generalization scenarios. That improvement is for fine-tuning that includes egocentric human data. The scaling observation is more important than the single number. Benefits from human data increase with the diversity of robot pretraining. Official visualizations show closer alignment of human and robot features with broader pretraining.

I see human data as an amplifier of a broad robot prior. It is not a cheap substitute for robot experience. For learning from human video, the human behavior must be relevant to the target body.

π*0.6 and RECAP: Learn from outcomes and corrections

π*0.6, published on November 17, 2025, introduces RECAP: Reinforcement Learning with Experience and Corrections via Advantage-conditioned Policies.

Deployed robots collect successes, failures and human corrections. Imitation of all behavior copies failures. Removal of all failures loses information about the errors.

RECAP uses experience quality as an input condition. A value function estimates which behavior made the outcome better. Policy training uses an advantage label such as:

Advantage: positive
Advantage: negative

At inference, the prompt can give a positive-advantage condition to the policy.

An edited draft gives an analogy. The version with an error shows the mistake. The correction shows a different behavior. The advantage label tells the policy which behavior was better.

RECAP is organized in three stages:

Offline RL pretraining: A value function trains from broad robot demonstrations. Then a π*0.6 policy trains with inferred advantage as a condition.
Task-specific supervised fine-tuning: Training adapts the policy to tasks such as laundry or box assembly with successful demonstrations as positive behavior.
Iterative improvement: The system deploys the policy and collects autonomous rollouts and interventions. It relabels the expanded dataset with the updated value function. Then it trains the policy again and deploys it again.

Fig. 6. RECAP’s iterative stage expands rollout-and-correction data and uses updated value estimates to relabel advantage. The advantage-conditioned policy trains again, and the system deploys it again. Offline pretraining and task-specific SFT come before this loop and have summaries above it. Full interactive figure

A failure with its correction can identify inferior behavior and train a steerable policy. It is not only a demonstration for imitation.

The reliability of advantage labels depends on value estimation and relabeling. RECAP lets the policy use that evaluation. An accurate evaluation is necessary.

OpenPI: Adapt policies to another robot and dataset

Open Sourcing π0 supplies code, checkpoints and a recipe to adapt policies to different robot bodies and datasets.

Each robot has cameras, calibration, action/state coordinates, gripper conventions, data formats and tasks. These parts must be adapted for reuse. The name “generalist” alone does not do that work.

The adaptation work in OpenPI has these parts:

Concern	Method-level role
Backbone reuse	Use broad visual-language representations. Do not train robot perception from the start again.
Action adaptation	Fine-tune the action expert and the robot-specific state/action interface.
Dataset portability	Make data loading, normalization, camera views and task labels configurable for each robot.

The design goal is a change of robot arm with no new learning of what “pick up the towel” means. Cameras and action coordinates change, but the semantic task stays recognizable. OpenPI uses the backbone again and adapts motor outputs. It also gives the data-loading/normalization work necessary for a different body.

OpenPI gives a structure for adaptation. In robotics, this integration frequently determines whether a policy becomes reusable beyond isolated demonstrations.

Comparison map: learning changes parameters, and supported runtime conditions steer a trained policy without the same parameter update. — Fig. 7. Training signals change policy parameters. Supported runtime conditions steer a trained policy for a particular task, history or desired state. Human transfer and each condition are method-specific.

Part 2: Inference conditions that resolve ambiguity

For a broad trained policy, “clean the table” can have context that is not sufficient. Which object is next? Which operations are completed? Which scene is the goal? Part 2 examines the subtasks, history and goals supplied for action selection.

Hi-Robot: Turn a request into language subtasks

Hi-Robot, published in February 2025, uses instructions such as “tidy this area” or “put the dishes away.” Instructions for each motor step from the user are not necessary.

Physical skill alone does not resolve interpretation. The robot must select the sequence, the objects that belong together, a response and a use for a correction.

Hi-Robot has a high-level VLM and a low-level π0-style VLA. The high-level model receives open-ended instructions, observations and possible human intervention. It can give verbal responses or simple language subtasks. The low-level policy executes those subtasks as physical movement.

Fig. 8. A high-level VLM interprets the instruction and interaction, then supplies a language subtask to a low-level VLA. The VLA also reads the current scene and robot state. Continuous execution gives new observations for the next high-level decision. Full interactive figure

“Clean the kitchen” becomes steps such as counter clearing, dish movement, food storage and surface wiping. Task decomposition is also necessary before low-level control can execute the request.

Hi-Robot uses language for task structure, clarification and correction. The low-level VLA controls continuous motion. The primary change is in interaction, not motor representation.

The hierarchy adds coordination. It can select an incorrect subtask or execute a good subtask poorly. I find a high-level interpretation stage valuable for open-ended instructions, with the motor controller at the low level.

A subtask becomes an explicit condition in π0.5

At inference, π0.5 first predicts a high-level language action. Then the current observation, original instruction and predicted subtask condition the continuous action expert. Hi-Robot uses two models. π0.5 uses one VLA for the two steps.

One compact representation is:

\pi_\theta(a_{t:t+H-1}, \hat{s} \mid o_t, l) \approx \pi_\theta(\hat{s} \mid o_t, l) \, \pi_\theta(a_{t:t+H-1} \mid o_t, l, \hat{s})

$o_t$ is the current observation, with relevant robot state. $l$ is the instruction. $\hat{s}$ is the predicted subtask, and $a_{t:t+H-1}$ is the continuous chunk. The representation gives the next language step and conditions action on that step and the current context.

Fig. 9. Author-redrawn inference view of π0.5. A language-level subtask conditions continuous action generation with the current observation and original instruction. The repeated context cards show data reuse, not two sensors. Source: Physical Intelligence π0.5 blog. Full interactive figure

π0.5 uses task context in control. It does not expect the instruction to determine each low-level command directly. It predicts a subtask and conditions continuous motion on that subtask.

A subtask gives the next operation, not the history. Completed-operation history and recent geometry are also necessary when objects become hidden.

MEM: Keep task progress and recent geometry separately

For a fifteen-minute cleaning task, a robot must remember completed cupboards, failed grasps and object locations during occlusion. A response to the current frame alone is not sufficient.

MEM: Multi-Scale Embodied Memory, released on March 3, 2026, supplies two kinds of memory to π0.6. Their functions are different and cannot be interchanged.

A longer context window does not select the history representation. Is text or image storage necessary? “The bowls are already in the cabinet” is task progress that language can represent. Visual geometry can be necessary for a partly hidden utensil, a folded cloth edge or a cup moved by contact.

MEM splits the problem:

Timescale	Representation	Used for
Long-term	Language memory summary	Progress, completed operations, high-level subtask selection
Short-term	Compact video memory	Occlusion, local geometry, corrective manipulation

Language memory summarizes progress for high-level decisions. After the robot stores three bowls, it must remember that they are there. It is not necessary to keep all placement frames.

Short-term video keeps local geometry for corrections such as adjustment of a failed grasp. Language can compress that geometry too much.

The official page gives results for tasks up to fifteen minutes long, partial-observability handling and in-context correction of manipulation strategies with MEM.

MEM selects the summary content, the visual content and the policy that reads each one. I find this more useful than one undifferentiated memory buffer.

π0.7: Condition on the desired future

π0.7, published April 16, 2026, supplies more conditions to steer a generalist robotic policy.

The task is “Put the food on the table”. It does not give grasp geometry, door angle, plate pose or the desired near-future scene.

π0.7 increases the information that a prompt can contain. A conventional prompt can contain:

Task: clean the kitchen
Subtask: open the fridge

The official release adds conditions for behavior quality, history and subgoal images. Robot data has fast/clumsy and slow/careful demonstrations, successes/failures, human-directed episodes and autonomous rollouts. Labels let the model distinguish regimes. A prompt can request one regime at inference, not an average of the regimes.

A Zhihu reading of π0.7 emphasizes that quality labels make mixed-quality data more usable.

The π0.7 architecture extends the context for a continuous-action VLA. The policy can use history, metadata and visual subgoals with observations and language.

Fig. 10. Author-redrawn conditioning view of π0.7. Training uses labeled instructions, subgoal images and episode metadata. Inference conditions include task/subtask context, observation memory, desired metadata and optional world-model subgoals in one supported mode. The prompt interface receives these conditions before continuous action generation. Visual subgoals are optional conditions. Source: Physical Intelligence π0.7 blog. Full interactive figure

One new condition is the subgoal image. In one inference mode, a lightweight world model generates visual subgoals from the current observation and subtask. Those images can then condition π0.7. Visual subgoals are one supported prompt modality.

Fig. 11. In one supported π0.7 inference mode, a lightweight world model uses the current observation and task context to supply a subgoal image as a proposal. The VLA conditions continuous action on the current scene and that desired target. Full interactive figure

Plan-view comparison: an orange object is outside a tray in the current scene and inside it in the desired visual subgoal. — Fig. 12. An orange object starts outside the tray. The visual subgoal gives its target position inside the tray. This plan-view scene gives a target condition.

The release gives results for complex-instruction following, counterfactual steering, cross-embodiment behavior and stronger execution with high-level subgoals. In its RECAP-style settings, π0.7 approaches or matches task-specific RL specialists.

A Reddit robotics discussion has questions about repeatability, noise, backlash, wear, friction and out-of-distribution failures. I read that skepticism as a demand for deployment tests.

In my reading, each prompt input gives something different. Language gives the task. Metadata requests a behavior regime, such as slow and careful. Control-modality labels give the method to read the action. Memory supplies history. Optional visual subgoals show the desired geometry.

The policy generates continuous actions, but a sentence is not its only condition.

A generated subgoal can also cause risk. An incorrect target can steer the VLA toward an incorrect future. An explicit target lets engineers examine the target, make it better and use it as an action condition.

Part 3: Execution in physical time

A good proposed chunk must arrive in time. Its precision must be sufficient for contact. Part 3 examines two problems that better prompts cannot repair. Inference can finish after robot movement changes the state. Millimeter precision can be necessary for contact.

Real-time chunking: Generate while the robot moves

Real-Time Action Chunking, released June 9, 2025, uses the overlap between execution and inference. The robot executes an old chunk while a large policy generates the next one. The state can change before inference finishes.

If inference latency is $\delta$ and the low-level controller period is $\Delta t_{\mathrm{ctrl}}$ , this number of control steps is committed:

d = \left\lfloor \frac{\delta}{\Delta t_{\mathrm{ctrl}}} \right\rfloor

The robot executes the first $d$ steps before the next chunk arrives. Thus, the new plan must start from those steps.

Synchronous inference can pause the robot between chunks. Naive asynchronous switching can conflict with motion in progress. Averaging overlapping chunks (temporal ensembling) blends two incompatible motions. It does not select one motion.

RTC uses inpainting for successor-chunk generation. The desired constraint makes the first $d$ actions agree with commands that will execute before inference finishes. The goal for the remaining overlap is to keep the previous strategy if it is useful. Later actions can change with the new observation.

Fig. 13. While the controller executes, RTC guides the next chunk toward prefix commands that are committed. Among eight illustrative positions, three prefix positions have weight one, two overlap positions decay and three new-future positions have zero weight. Guidance is approximate, not a hard prefix overwrite. Full interactive figure

The flow-matching representation gives the position of that constraint in generation. Each integration step predicts a final chunk. RTC adds gradient guidance toward the known prefix and a weighted overlap target. Prefix weights are one. Overlapping future weights decay. New nonoverlapping actions have zero weight.

The implementation approximates the desired constraint with guidance, not a hard prefix overwrite. The prefix can have a nonzero error. The implementation clips guidance because a strong correction with few integration steps can destabilize the trajectory.

With joint generation, the prefix can change the remaining motion. Soft overlap decreases strategy changes. Hard-mask comparisons and guidance ablations give evidence for those selections. RTC operates without policy training but adds inference-time guidance work. One worker executes actions, and another generates actions.

RTC uses the overlap between execution and inference to keep delayed decisions consistent with motion. RTC hides inference delay behind execution; the model’s inference time is unchanged.

RLT: Refine a VLA proposal with local online learning

RLT: Precise Manipulation with Efficient Online RL, published on March 19, 2026, is for a different limitation. A generalist VLA can be good enough to approach a task but insufficiently precise for the contact-rich final phase. Examples are zip-tie fastening, screw insertion and connection of an Ethernet connector.

Online updates to the full VLA are expensive and risky. The model is large, and its training can have sparse rewards. Only a narrow range of contact exploration is useful.

RLT uses the VLA as a representation and behavior prior. A smaller online learner makes local behavior better. The objective below is a schematic view of regularized improvement. The refinement uses reward and a penalty for movement away from the VLA proposal:

\max_a \; Q(x, a) - \lambda \lVert a - a_{\mathrm{VLA}} \rVert_2^2

The goal is to make local action better and keep it close to the VLA proposal.

The VLA supplies task understanding, scene context and a broad manipulation strategy. A smaller learner is for the last difficult millimeters where contact and precision dominate.

The VLA has three functions: representation, behavior prior and a constraint on risky exploration. The official RLT release gives results for fast online improvement on tasks that include zip-tie fastening and screw insertion.

RLT is for precision phases with a strong prior. I read it as local refinement, not online RL that replaces a general robot foundation model.

What I take from the open-pi sequence

I read these works less as contests between tokens/flow, hierarchy/end-to-end learning or imitation/RL. I read them more as choices about which representation helps learning, inference or execution. These are different functions.

Action tokens can be policy outputs or targets for action-structure training. Flow matching generates continuous motor outputs. Hierarchy selects subtasks. KI controls which losses update the backbone.

RTC keeps a delayed chunk consistent with motion during inference. Advantage labels supply experience-quality information. Broad robot priors make human transfer possible.

MEM uses text for task progress and video for recent history. RLT makes a VLA proposal better. Visual subgoals give desired scenes.

These selections change different things. FAST changes a target, and KI changes gradient routing. Subtasks/memory/goals condition inference, and RTC constrains a successor chunk. The term “richer prompting” does not identify those different decisions.

π0.7 combines several of these functions, but the research problem stays open. Continuous output lets a VLM act. More conditions steer the model. The execution loop must keep continuity. Separate tests of repeatable task completion are also necessary.

Do a separate test for each type of change. I recommend these design goals:

A training signal makes transfer better and keeps semantics.
An inference condition resolves ambiguity or makes recovery better.
An execution method keeps continuity and responsiveness with measured delay/contact.

A pass in one test does not show a pass in the others.

My foundation-model goal is a robot that engineers can examine and correct. Its design joins perception, language, action, memory, feedback and prediction. Open-pi gives several of these decisions. More work is necessary to join them in one robot. The Robot-use agents article gives a different method for high-level reasoning and physical execution.

Technical reference notes

The official releases below give different engineering decisions in one model family. Use this reference table to find a mechanism and its source.

Work	Core interface	Function
π0	VLM backbone plus flow-matching action expert	π0 adds a continuous motor-output path to pretrained visual-language representations.
OpenPI	Open-source code, checkpoints and fine-tuning recipe	OpenPI helps engineers use π0-style policies again with different robot setups.
FAST / FAST+	Frequency-structured action tokens	FAST compresses smooth action chunks for autoregressive action-token policies and FAST-tokenized representation learning.
Hi-Robot	High-level VLM plus low-level VLA	Hi-Robot generates executable subtasks from open-ended instructions and feedback.
π0.5	Heterogeneous co-training and subtask-conditioned action	π0.5 uses web data, multi-environment robot data, cross-embodiment data and high-level labels for open-world generalization.
Knowledge Insulation	Stop-gradient between backbone and action expert	The backbone adapts to robot-relevant structure. Continuous control loss cannot cause damage to its semantic knowledge through that gradient path.
RTC	Inpainted successor chunks with latency	RTC keeps chunked action generation compatible with physical execution while the robot moves.
π*0.6 / RECAP	Advantage-conditioned policy from experience and corrections	RECAP supplies deployment outcomes as a condition that the policy can use.
Human2Robot	Co-training with egocentric human demonstrations	Human behavior is an amplifier after broad robot pretraining creates a transferable prior.
MEM	Long-term text for task progress and short-term memory for visual observations	MEM keeps task progress in text and recent visual geometry in video.
RLT	Small specialist for online RL with a VLA prior	RLT makes contact-rich precision phases better. It does not train the full foundation policy again online.
π0.7	Structured steering with metadata conditioning, memory, control modalities and visual subgoals	A continuous-action policy can use more conditions. These include world-model-generated subgoals in one supported inference mode.

Citation

Use this citation:

Ji, Wenbo. “From π0 to π0.7: A Tutorial on Open-pi and Robot Foundation Models”. fusheng-ji.github.io (May 2026). https://fusheng-ji.github.io/blog/posts/from-pi0-to-pi07-open-pi-series/

Or use the BibTeX entry:

@article{ji2026frompi0topi07openpiseries,
  title = {From π0 to π0.7: A Tutorial on Open-pi and Robot Foundation Models},
  author = {Ji, Wenbo},
  journal = {fusheng-ji.github.io},
  year = {2026},
  month = {May},
  url = {https://fusheng-ji.github.io/blog/posts/from-pi0-to-pi07-open-pi-series/}
}

References

Robotics

Updated 08 November 2026