# Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interaction

arXiv: https://arxiv.org/abs/2410.03920 · HTML: https://mlsanity.com/paper/2410.03920/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 67 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification](https://mlsanity.com/paper/2303.05512/) (2023)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [3D Neural Scene Representations for Visuomotor Control](https://mlsanity.com/paper/2107.04004/) (2021)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics](https://mlsanity.com/paper/2104.03311/) (2021)
- [ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics](https://mlsanity.com/paper/1810.01054/) (2018)
- [NeuralSim: Augmenting Differentiable Simulators with Neural Networks](https://mlsanity.com/paper/2011.04217/) (2020)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [DiffPD: Differentiable Projective Dynamics](https://mlsanity.com/paper/2101.05917/) (2021)
- [Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics](https://mlsanity.com/paper/2304.14369/) (2023)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [Visual Grounding of Learned Physical Models](https://mlsanity.com/paper/2004.13664/) (2020)
- [Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contact](https://mlsanity.com/paper/2103.16021/) (2021)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Differentiable Simulation of Soft Multi-body Systems](https://mlsanity.com/paper/2205.01758/) (2022)
- [Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objects](https://mlsanity.com/paper/2203.00663/) (2022)
- [RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation](https://mlsanity.com/paper/2205.05678/) (2022)
- [Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2011.01734/) (2020)
- [Encoding Physical Constraints in Differentiable Newton-Euler Algorithm](https://mlsanity.com/paper/2001.08861/) (2020)
- [Interactive Differentiable Simulation](https://mlsanity.com/paper/1905.10706/) (2019)
- [Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation](https://mlsanity.com/paper/2109.08815/) (2021)
- [Model-Based Control with Sparse Neural Dynamics](https://mlsanity.com/paper/2312.12791/) (2023)
- [Inferring Articulated Rigid Body Dynamics from RGBD Video](https://mlsanity.com/paper/2203.10488/) (2022)
- [Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots](https://mlsanity.com/paper/2303.09597/) (2023)
- [A Differentiable Newton-Euler Algorithm for Real-World Robotics](https://mlsanity.com/paper/2110.12422/) (2021)

## Built on by (most cited first, 5 of 5)

- [D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping](https://mlsanity.com/paper/2603.01151/) (2026)
- [HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation](https://mlsanity.com/paper/2603.15084/) (2026)
- [NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception](https://mlsanity.com/paper/2607.11734/) (2026)
- [FACT: Fidelity-Aware Construction of Articulated Twins](https://mlsanity.com/paper/2609.37067/) (2026)
- [DexNinja: Learning Robust Dexterous Cutting Policy with a Real-to-Sim-to-Real Data Engine](https://mlsanity.com/paper/2610.17208/) (2026)

## Abstract (arXiv)

Differentiable simulation has become a powerful tool for system identification. While prior work has focused on identifying robot properties using robot-specific data or object properties using object-specific data, our approach calibrates object properties by using information from the robot, without relying on data from the object itself. Specifically, we utilize robot joint encoder information, which is commonly available in standard robotic systems. Our key observation is that by analyzing the robot's reactions to manipulated objects, we can infer properties of those objects, such as inertia and softness. Leveraging this insight, we develop differentiable simulations of robot-object interactions to inversely identify the properties of the manipulated objects. Our approach relies solely on proprioception -- the robot's internal sensing capabilities -- and does not require external measurement tools or vision-based tracking systems. This general method is applicable to any articulated robot and requires only joint position information. We demonstrate the effectiveness of our method on a low-cost robotic platform, achieving accurate mass and elastic modulus estimations of manipulated objects with just a few seconds of computation on a laptop.