# Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements

arXiv: https://arxiv.org/abs/2404.07344 · HTML: https://mlsanity.com/paper/2404.07344/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 10 Apr 2024.

## Builds on (most cited first, 9 of 9)

- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [ComPhy: Compositional Physical Reasoning of Objects and Events from Videos](https://mlsanity.com/paper/2205.01089/) (2022)
- [SHOP-VRB: A Visual Reasoning Benchmark for Object Perception](https://mlsanity.com/paper/2004.02673/) (2020)
- [From Multi-modal Property Dataset to Robot-centric Conceptual Knowledge About Household Objects](https://mlsanity.com/paper/1906.11114/) (2019)
- [Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images](https://mlsanity.com/paper/2004.14487/) (2020)
- [Probabilistic Surface Friction Estimation Based on Visual and Haptic Measurements](https://mlsanity.com/paper/2010.08277/) (2020)
- [Push to know! -- Visuo-Tactile based Active Object Parameter Inference with Dual Differentiable Filtering](https://mlsanity.com/paper/2308.01001/) (2023)
- [An Empirical Evaluation of Various Information Gain Criteria for Active Tactile Action Selection for Pose Estimation](https://mlsanity.com/paper/2205.04697/) (2022)
- [Online Elasticity Estimation and Material Sorting Using Standard Robot Grippers](https://mlsanity.com/paper/2401.08298/) (2024)

## Built on by (most cited first, 5 of 5)

- [Closed Loop Interactive Embodied Reasoning for Robot Manipulation](https://mlsanity.com/paper/2404.15194/) (2024)
- [Task-oriented Robotic Manipulation with Vision Language Models](https://mlsanity.com/paper/2410.15863/) (2024)
- [CA-World: Multi-Object Counterfactual Alignment for Efficient Interactive-Ready Reconstruction](https://mlsanity.com/paper/2605.30239/) (2026)
- [Learning to Double Guess: An Active Perception Approach for Estimating the Center of Mass of Arbitrary Objects](https://mlsanity.com/paper/2502.02663/) (2025)
- [Occlusion-Robust Multi-Object Decoupling for Physics-Based Robotic Interaction](https://mlsanity.com/paper/2606.29303/) (2026)

## Abstract (arXiv)

This work presents a framework for automatically extracting physical object properties, such as material composition, mass, volume, and stiffness, through robot manipulation and a database of object measurements. The framework involves exploratory action selection to maximize learning about objects on a table. A Bayesian network models conditional dependencies between object properties, incorporating prior probability distributions and uncertainty associated with measurement actions. The algorithm selects optimal exploratory actions based on expected information gain and updates object properties through Bayesian inference. Experimental evaluation demonstrates effective action selection compared to a baseline and correct termination of the experiments if there is nothing more to be learned. The algorithm proved to behave intelligently when presented with trick objects with material properties in conflict with their appearance. The robot pipeline integrates with a logging module and an online database of objects, containing over 24,000 measurements of 63 objects with different grippers. All code and data are publicly available, facilitating automatic digitization of objects and their physical properties through exploratory manipulations.