# Understanding Physical Effects for Effective Tool-use

arXiv: https://arxiv.org/abs/2206.14998 · HTML: https://mlsanity.com/paper/2206.14998/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 15 of 15)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Synthesizing Diverse and Physically Stable Grasps with Arbitrary Hand Structures using Differentiable Force Closure Estimator](https://mlsanity.com/paper/2104.09194/) (2021)
- [KETO: Learning Keypoint Representations for Tool Manipulation](https://mlsanity.com/paper/1910.11977/) (2019)
- [DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting](https://mlsanity.com/paper/2105.12244/) (2021)
- [Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning](https://mlsanity.com/paper/1907.09620/) (2019)
- [Codimensional Incremental Potential Contact](https://mlsanity.com/paper/2012.04457/) (2020)
- [Inferring the Material Properties of Granular Media for Robotic Tasks](https://mlsanity.com/paper/2003.08032/) (2020)
- [GIFT: Generalizable Interaction-aware Functional Tool Affordances without Labels](https://mlsanity.com/paper/2106.14973/) (2021)
- [ADD: Analytically Differentiable Dynamics for Multi-Body Systems with Frictional Contact](https://mlsanity.com/paper/2007.00987/) (2020)
- [How to select and use tools? : Active Perception of Target Objects Using Multimodal Deep Learning](https://mlsanity.com/paper/2106.02445/) (2021)
- [Efficient Task Planning for Mobile Manipulation: a Virtual Kinematic Chain Perspective](https://mlsanity.com/paper/2108.01259/) (2021)
- [Consolidating Kinematic Models to Promote Coordinated Mobile Manipulations](https://mlsanity.com/paper/2108.01264/) (2021)
- [Tool Macgyvering: Tool Construction Using Geometric Reasoning](https://mlsanity.com/paper/1902.03666/) (2019)
- [TANGO: Commonsense Generalization in Predicting Tool Interactions for Mobile Manipulators](https://mlsanity.com/paper/2105.04556/) (2021)
- [Soft Hybrid Aerial Vehicle via Bistable Mechanism](https://mlsanity.com/paper/2011.00426/) (2020)

## Built on by (most cited first, 7 of 7)

- [I-PHYRE: Interactive Physical Reasoning](https://mlsanity.com/paper/2312.03009/) (2023)
- [On the Learning Mechanisms in Physical Reasoning](https://mlsanity.com/paper/2210.02075/) (2022)
- [STRAP: Structured Object Affordance Segmentation with Point Supervision](https://mlsanity.com/paper/2304.08492/) (2023)
- [Learning Generalizable Tool-use Skills through Trajectory Generation](https://mlsanity.com/paper/2310.00156/) (2023)
- [M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes](https://mlsanity.com/paper/2410.06678/) (2024)
- [Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulation](https://mlsanity.com/paper/2503.03556/) (2025)
- [Prolonging Tool Life: Learning Skillful Use of General-purpose Tools through Lifespan-guided Reinforcement Learning](https://mlsanity.com/paper/2507.17275/) (2025)

## Abstract (arXiv)

We present a robot learning and planning framework that produces an effective tool-use strategy with the least joint efforts, capable of handling objects different from training. Leveraging a Finite Element Method (FEM)-based simulator that reproduces fine-grained, continuous visual and physical effects given observed tool-use events, the essential physical properties contributing to the effects are identified through the proposed Iterative Deepening Symbolic Regression (IDSR) algorithm. We further devise an optimal control-based motion planning scheme to integrate robot- and tool-specific kinematics and dynamics to produce an effective trajectory that enacts the learned properties. In simulation, we demonstrate that the proposed framework can produce more effective tool-use strategies, drastically different from the observed ones in two exemplar tasks.