# ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

arXiv: https://arxiv.org/abs/2609.21751 · HTML: https://mlsanity.com/paper/2609.21751/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 28 of 28)

- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Ditto: Building Digital Twins of Articulated Objects from Interaction](https://mlsanity.com/paper/2202.08227/) (2022)
- [Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation](https://mlsanity.com/paper/2403.03949/) (2024)
- [PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objects](https://mlsanity.com/paper/2308.07391/) (2023)
- [URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images](https://mlsanity.com/paper/2405.11656/) (2024)
- [Automated Creation of Digital Cousins for Robust Policy Learning](https://mlsanity.com/paper/2410.07408/) (2024)
- [PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos](https://mlsanity.com/paper/2503.17973/) (2025)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [A Probabilistic Framework for Learning Kinematic Models of Articulated Objects](https://mlsanity.com/paper/1405.7705/) (2014)
- [OPDMulti: Openable Part Detection for Multiple Objects](https://mlsanity.com/paper/2303.14087/) (2023)
- [Articulated Object Estimation in the Wild](https://mlsanity.com/paper/2509.01708/) (2025)
- [In-the-Wild Compliant Manipulation with UMI-FT](https://mlsanity.com/paper/2601.09988/) (2026)
- [Articulated 3D Scene Graphs for Open-World Mobile Manipulation](https://mlsanity.com/paper/2602.16356/) (2026)
- [Sim2Real2: Actively Building Explicit Physics Model for Precise Articulated Object Manipulation](https://mlsanity.com/paper/2302.10693/) (2023)
- [Online Estimation of Articulated Objects with Factor Graphs using Vision and Proprioceptive Sensing](https://mlsanity.com/paper/2309.16343/) (2023)
- [Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation](https://mlsanity.com/paper/2410.13817/) (2024)
- [Hoi! - A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation](https://mlsanity.com/paper/2512.04884/) (2025)
- [FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos](https://mlsanity.com/paper/2604.05621/) (2026)
- [REACT3D: Recovering Articulations for Interactive Physical 3D Scenes](https://mlsanity.com/paper/2510.11340/) (2025)
- [Pandora: Articulated 3D Scene Graphs from Egocentric Vision](https://mlsanity.com/paper/2603.28732/) (2026)
- [ArtiSG: Functional 3D Scene Graph Construction via Human-demonstrated Articulated Objects Manipulation](https://mlsanity.com/paper/2512.24845/) (2025)
- [FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning](https://mlsanity.com/paper/2604.03696/) (2026)
- [Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objects](https://mlsanity.com/paper/2605.24403/) (2026)
- [Online Estimation and Manipulation of Articulated Objects](https://mlsanity.com/paper/2601.01438/) (2026)
- [PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning](https://mlsanity.com/paper/2606.08655/) (2026)

## Abstract (arXiv)

Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/