# ContactGaussian-WM: Learning Physics-Grounded World Model from Videos

arXiv: https://arxiv.org/abs/2602.11021 · HTML: https://mlsanity.com/paper/2602.11021/

## Key facts

- Cited by 7 papers in the map (data through 7 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 19 of 19)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators](https://mlsanity.com/paper/1906.01728/) (2019)
- [LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning](https://mlsanity.com/paper/2309.06440/) (2023)
- [ASID: Active Exploration for System Identification in Robotic Manipulation](https://mlsanity.com/paper/2404.12308/) (2024)
- [Dojo: A Differentiable Physics Engine for Robotics](https://mlsanity.com/paper/2203.00806/) (2022)
- [PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation](https://mlsanity.com/paper/2504.16693/) (2025)
- [DSSIM: a structural similarity index for floating-point data](https://mlsanity.com/paper/2202.02616/) (2022)
- [QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer](https://mlsanity.com/paper/2404.07988/) (2024)
- [TwinAligner: Visual-Dynamic Alignment Empowers Physics-aware Real2Sim2Real for Robotic Manipulation](https://mlsanity.com/paper/2512.19390/) (2025)
- [Splatting Physical Scenes: End-to-End Real-to-Sim from Imperfect Robot Data](https://mlsanity.com/paper/2506.04120/) (2025)
- [TwinTrack: Bridging Vision and Contact Physics for Real-Time Tracking of Unknown Objects in Contact-Rich Scenes](https://mlsanity.com/paper/2505.22882/) (2025)

## Built on by (most cited first, 7 of 7)

- [World2Act: Latent Action Post-Training from World Model Dynamics](https://mlsanity.com/paper/2603.10422/) (2026)
- [IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video](https://mlsanity.com/paper/2603.16432/) (2026)
- [From World Models to World Action Models: A Concise Tutorial for Robotics](https://mlsanity.com/paper/2607.00836/) (2026)
- [Robots Need More than VLA and World Models](https://mlsanity.com/paper/2606.06556/) (2026)
- [Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects](https://mlsanity.com/paper/2606.01950/) (2026)
- [LaGSplat: Inferring Physics-Governed Interactive Simulation from Monocular Video Using Latent Lagrangian Gaussian Splatting](https://mlsanity.com/paper/2608.16324/) (2026)
- [Learning Collective Dynamics with Differentiable Gaussian Representations](https://mlsanity.com/paper/2609.28405/) (2026)

## Abstract (arXiv)

Developing world models that understand complex physical interactions is essential for advancing robotic planning and simulation.However, existing methods often struggle to accurately model the environment under conditions of data scarcity and complex contact-rich dynamic motion.To address these challenges, we propose ContactGaussian-WM, a differentiable physics-grounded rigid-body world model capable of learning intricate physical laws directly from sparse and contact-rich video sequences.Our framework consists of two core components: (1) a unified Gaussian representation for both visual appearance and collision geometry, and (2) an end-to-end differentiable learning framework that differentiates through a closed-form physics engine to infer physical properties from sparse visual observations.Extensive simulations and real-world evaluations demonstrate that ContactGaussian-WM outperforms state-of-the-art methods in learning complex scenarios, exhibiting robust generalization capabilities.Furthermore, we showcase the practical utility of our framework in downstream applications, including data synthesis and real-time MPC.