# Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planning

arXiv: https://arxiv.org/abs/1904.10348 · HTML: https://mlsanity.com/paper/1904.10348/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2019.

## Built on by (most cited first, 24 of 24)

- [Visual Room Rearrangement](https://mlsanity.com/paper/2103.16544/) (2021)
- [LEGO-Net: Learning Regular Rearrangements of Objects in Rooms](https://mlsanity.com/paper/2301.09629/) (2023)
- [IFOR: Iterative Flow Minimization for Robotic Object Rearrangement](https://mlsanity.com/paper/2202.00732/) (2022)
- [NeRP: Neural Rearrangement Planning for Unknown Objects](https://mlsanity.com/paper/2106.01352/) (2021)
- [Semantically Grounded Object Matching for Robust Robotic Scene Rearrangement](https://mlsanity.com/paper/2111.07975/) (2021)
- [Monte Carlo Scene Search for 3D Scene Understanding](https://mlsanity.com/paper/2103.07969/) (2021)
- [CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation](https://mlsanity.com/paper/2304.09302/) (2023)
- [Transporters with Visual Foresight for Solving Unseen Rearrangement Tasks](https://mlsanity.com/paper/2202.10765/) (2022)
- [Coarse-to-fine Q-attention with Tree Expansion](https://mlsanity.com/paper/2204.12471/) (2022)
- [Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planning](https://mlsanity.com/paper/2309.02547/) (2023)
- [Efficient and High-quality Prehensile Rearrangement in Cluttered and Confined Spaces](https://mlsanity.com/paper/2110.02814/) (2021)
- [A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search](https://mlsanity.com/paper/2206.13396/) (2022)
- [Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior](https://mlsanity.com/paper/2402.15402/) (2024)
- [Tree-Guided Diffusion Planner](https://mlsanity.com/paper/2508.21800/) (2025)
- [ORLA*: Mobile Manipulator-Based Object Rearrangement with Lazy A Star](https://mlsanity.com/paper/2309.13707/) (2023)
- [Hierarchical Vision Language Action Model Using Success and Failure Demonstrations](https://mlsanity.com/paper/2512.03913/) (2025)
- [Assembly Planning from Observations under Physical Constraints](https://mlsanity.com/paper/2204.09616/) (2022)
- [Simultaneous Navigation and Construction Benchmarking Environments](https://mlsanity.com/paper/2103.16732/) (2021)
- [PEARL: Parallelized Expert-Assisted Reinforcement Learning for Scene Rearrangement Planning](https://mlsanity.com/paper/2105.04088/) (2021)
- [MANER: Multi-Agent Neural Rearrangement Planning of Objects in Cluttered Environments](https://mlsanity.com/paper/2306.06543/) (2023)
- [Robotic Test Tube Rearrangement Using Combined Reinforcement Learning and Motion Planning](https://mlsanity.com/paper/2401.09772/) (2024)
- [Enhancing Bayesian Network Structural Learning with Monte Carlo Tree Search](https://mlsanity.com/paper/2502.01527/) (2025)
- [Multi-step manipulation task and motion planning guided by video demonstration](https://mlsanity.com/paper/2505.08949/) (2025)
- [Dynamic Buffers: Cost-Efficient Planning for Tabletop Rearrangement with Stacking](https://mlsanity.com/paper/2509.22828/) (2025)

## Abstract (arXiv)

We address the problem of visually guided rearrangement planning with many movable objects, i.e., finding a sequence of actions to move a set of objects from an initial arrangement to a desired one, while relying on visual inputs coming from an RGB camera. To do so, we introduce a complete pipeline relying on two key contributions. First, we introduce an efficient and scalable rearrangement planning method, based on a Monte-Carlo Tree Search exploration strategy. We demonstrate that because of its good trade-off between exploration and exploitation our method (i) scales well with the number of objects while (ii) finding solutions which require a smaller number of moves compared to the other state-of-the-art approaches. Note that on the contrary to many approaches, we do not require any buffer space to be available. Second, to precisely localize movable objects in the scene, we develop an integrated approach for robust multi-object workspace state estimation from a single uncalibrated RGB camera using a deep neural network trained only with synthetic data. We validate our multi-object visually guided manipulation pipeline with several experiments on a real UR-5 robotic arm by solving various rearrangement planning instances, requiring only 60 ms to compute the plan to rearrange 25 objects. In addition, we show that our system is insensitive to camera movements and can successfully recover from external perturbations. Supplementary video, source code and pre-trained models are available at https://ylabbe.github.io/rearrangement-planning.