μ0: A Scalable 3D Interaction-Trace World Model
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World models that capture how actions induce physical change enable scalable robot learning without reliance on embodiment-specific action labels. Pixel-space video models provide broad visual priors but expend model capacity on dense appearance reconstruction, while direct action models require embodiment-specific labels that hinder scalability. We present μ0, a scalable world model based on 3D traces. Rather than predicting dense pixels or directly modeling actions, μ0 forecasts smooth 3D trajectories for salient interaction points such as objects, tools, hands, and contact regions, yielding a compact, embodiment-agnostic motion interface. To enable training from diverse video sources, our TraceExtract system automatically extracts 3D supervision by selecting keypoints, constructing globally aligned traces, and associating motion segments with hierarchical language captions. This TraceExtract supervision pretrains μ0 by combining a pretrained vision-language backbone with a modular trace expert, which represents each query via B-spline control points and predicts future traces. Experiments show that μ0 outperforms baselines in both 2D and 3D trace prediction, including trace prediction models and tokenized VLM methods. Because μ0 is frozen and reusable, it can be paired with action experts for downstream robot embodiments. Despite action-free pretraining, the resulting trace-conditioned policies achieve performance competitive with VLA models pretrained with action supervision, such as π0. These results establish 3D traces as a scalable and transferable representation for cross-embodiment manipulation.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 52 of its 55 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
52 of 55 references.
Closest ancestors · 13
Same direction, newest first
- 2026RoboFlow4D: A Lightweight Flow World Model Toward Real-Time Flow-Guided Robotic Manipulation17 May
- 2026EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards18 Mar
- 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Model18 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2026MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation10 Feb
- 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos6 Feb
- 2026Causal World Modeling for Robot Control29 Jan
- 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation7 Jan
- 2025TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videos26 Nov
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulation11 Oct
- 20253DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model6 Jun
- 2024FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model11 Dec
Other references · 39
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2024Depth Anything V2cited by 1,422
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
27 more
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
- 2023Any-point Trajectory Modeling for Policy Learningcited by 215
- 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 153
- 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
- 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
- 2024Flow as the Cross-Domain Manipulation Interfacecited by 132
- 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 130
- 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 124
- 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
- 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 85
- 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robotscited by 79
- 2026LeRobot: An Open-Source Library for End-to-End Robot Learningcited by 73
- 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 69
- 2025NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videoscited by 40
- 2025Trace Anything: Representing Any Video in 4D via Trajectory Fieldscited by 33
- 2025Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flowcited by 25
- 2025Towards Spatial Trace with Reasoning in Vision-Language Models for Roboticscited by 12
- 2026LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Priorcited by 11
- 2026Pri4R: Learning World Dynamics for Vision-Language-Action Models with Privileged 4D Representationcited by 8
- 2025Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognitioncited by 7
- 2025Pixel Motion Diffusion is What We Need for Robot Controlcited by 7
- 2026Forecasting Motion in the Wildcited by 5
- 2026LILAC: Language-Conditioned Object-Centric Optical Flow for Open-Loop Trajectory Generationcited by 4
- 2026EgoAVFlow: Robot Policy Learning with Active Vision from Human Egocentric Videos via 3D Flowcited by 3
- 2026LACE: Latent Visual Representation for Cross-Embodiment Learningcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026UNITAS: A 3D-Native World Action Model for Embodied Manipulation
- 2026Learning Transferable Dynamics Priors from Action to World Modeling
- 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- 2026PointAction: 3D Points as Universal Action Representations for Robot Control
- 2026Learning Transferable World-Action Models from Task-Paired Human Videos
- 2025ISS Policy : Scalable Diffusion Policy with Implicit Scene Supervision
- 2026Masked Visual Actions for Unified World Modeling
- 2026Foresight Without Seeing: Latent Futures for World Action Models