Mirage2Matter: A Physically Grounded Gaussian World Model from Video
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
The scalability of embodied intelligence is fundamentally constrained by the scarcity of real-world interaction data. While simulation platforms provide a promising alternative, existing approaches often suffer from a substantial visual and physical gap to real environments and rely on expensive sensors, precise robot calibration, or depth measurements, limiting their practicality at scale. We present Simulate Anything, a graphics-driven world modeling and simulation framework that enables efficient generation of high-fidelity embodied training data using only multi-view environment videos and off-the-shelf assets. Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS), seamlessly capturing fine-grained geometry and appearance from video. We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target, enabling accurate scale alignment between the reconstructed scene and the real world. Together, these components provide a unified, editable, and physically grounded world model. Vision Language Action (VLA) models trained on our simulated data achieve strong zero-shot performance on downstream tasks, matching or even surpassing results obtained with real-world data, highlighting the potential of reconstruction-driven world modeling for scalable and practical embodied intelligence training.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 21 of its 52 references that are in the map. First posted on arXiv 24 Jan 2026.
Builds on
21 of 52 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
- 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
- 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 171
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
- 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
- 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 60
- 2024RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulatorcited by 50
8 more
- 2025Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulationcited by 38
- 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
- 2025PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policiescited by 29
- 2025GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulationcited by 16
- 2025DISCOVERSE: Efficient Robot Simulation in Complex High-Fidelity Environmentscited by 12
- 2025Learning 3D-Gaussian Simulators from RGB Videoscited by 8
- 2025PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attentioncited by 4
- 2025Scan, Materialize, Simulate: A Generalizable Framework for Physically Grounded Robot Planningcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- 2024Physically Embodied Gaussian Splatting: A Realtime Correctable World Model for Robotics
- 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
- 2025High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning with Gaussian Splatting
- 2025RoboScape: Physics-informed Embodied World Model
- 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions
- 2026ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models
- 2025Simulating the Visual World with Artificial Intelligence: A Roadmap