# Vysics: Object Reconstruction Under Occlusion by Fusing Vision and Contact-Rich Physics

arXiv: https://arxiv.org/abs/2504.18719 · HTML: https://mlsanity.com/paper/2504.18719/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2025.

## Built on by (most cited first, 9 of 9)

- [Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions](https://mlsanity.com/paper/2511.04665/) (2025)
- [TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation](https://mlsanity.com/paper/2603.12665/) (2026)
- [Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation](https://mlsanity.com/paper/2510.11689/) (2025)
- [Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling](https://mlsanity.com/paper/2602.08058/) (2026)
- [Simulation-Ready Cluttered Scene Estimation via Physics-aware Joint Shape and Pose Optimization](https://mlsanity.com/paper/2602.20150/) (2026)
- [CHyLL: Learning Continuous Neural Representations of Hybrid Systems](https://mlsanity.com/paper/2512.10117/) (2025)
- [Object Reconstruction under Occlusion with Generative Priors and Contact-induced Constraints](https://mlsanity.com/paper/2512.05079/) (2025)
- [UNIC: Learning Unified Multimodal Extrinsic Contact Estimation](https://mlsanity.com/paper/2601.04356/) (2026)
- [PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splatting](https://mlsanity.com/paper/2606.03479/) (2026)

## Abstract (arXiv)

We introduce Vysics, a vision-and-physics framework for a robot to build an expressive geometry and dynamics model of a single rigid body, using a seconds-long RGBD video and the robot's proprioception. While the computer vision community has built powerful visual 3D perception algorithms, cluttered environments with heavy occlusions can limit the visibility of objects of interest. However, observed motion of partially occluded objects can imply physical interactions took place, such as contact with a robot or the environment. These inferred contacts can supplement the visible geometry with "physible geometry," which best explains the observed object motion through physics. Vysics uses a vision-based tracking and reconstruction method, BundleSDF, to estimate the trajectory and the visible geometry from an RGBD video, and an odometry-based model learning method, Physics Learning Library (PLL), to infer the "physible" geometry from the trajectory through implicit contact dynamics optimization. The visible and "physible" geometries jointly factor into optimizing a signed distance function (SDF) to represent the object shape. Vysics does not require pretraining, nor tactile or force sensors. Compared with vision-only methods, Vysics yields object models with higher geometric accuracy and better dynamics prediction in experiments where the object interacts with the robot and the environment under heavy occlusion. Project page: https://vysics-vision-and-physics.github.io/