# Planning with Selective Physics-based Simulation for Manipulation Among Movable Objects

arXiv: https://arxiv.org/abs/2003.06743 · HTML: https://mlsanity.com/paper/2003.06743/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2020.

## Built on by (most cited first, 4 of 4)

- [Planning for Manipulation among Movable Objects: Deciding Which Objects Go Where, in What Order, and How](https://mlsanity.com/paper/2303.13385/) (2023)
- [MOSAIC: Skill-Centric Manipulation Planning with Physics Simulation](https://mlsanity.com/paper/2504.16738/) (2025)
- [Planning for Complex Non-prehensile Manipulation Among Movable Objects by Interleaving Multi-Agent Pathfinding and Physics-Based Simulation](https://mlsanity.com/paper/2303.13352/) (2023)
- [Planning-Query-Guided Model Generation for Model-Based Deformable Object Manipulation](https://mlsanity.com/paper/2508.19199/) (2025)

## Abstract (arXiv)

Use of physics-based simulation as a planning model enables a planner to reason and generate plans that involve non-trivial interactions with the world. For example, grasping a milk container out of a cluttered refrigerator may involve moving a robot manipulator in between other objects, pushing away the ones that are movable and avoiding interactions with certain fragile containers. A physics-based simulator allows a planner to reason about the effects of interactions with these objects and to generate a plan that grasps the milk container successfully. The use of physics-based simulation for planning however is underutilized. One of the reasons for it being that physics-based simulations are typically way too slow for being used within a planning loop that typically requires tens of thousands of actions to be evaluated within a matter of a second or two. In this work, we develop a planning algorithm that tries to address this challenge. In particular, it builds on the observation that only a small number of actions actually need to be simulated using physics, and the remaining set of actions, such as moving an arm around obstacles, can be evaluated using a much simpler internal planning model, e.g., a simple collision-checking model. Motivated by this, we develop an algorithm called Planning with Selective Physics-based Simulation that automatically discovers what should be simulated with physics and what can utilize an internal planning model for pick-and-place tasks.