# Hierarchical Object-Oriented POMDP Planning for Object Rearrangement

arXiv: https://arxiv.org/abs/2412.01348 · HTML: https://mlsanity.com/paper/2412.01348/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 29 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation](https://mlsanity.com/paper/2007.04954/) (2020)
- [Rearrangement: A Challenge for Embodied AI](https://mlsanity.com/paper/2011.01975/) (2020)
- [PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning](https://mlsanity.com/paper/1802.08705/) (2018)
- [Visual Room Rearrangement](https://mlsanity.com/paper/2103.16544/) (2021)
- [PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks](https://mlsanity.com/paper/2411.00081/) (2024)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [Multi-skill Mobile Manipulation for Object Rearrangement](https://mlsanity.com/paper/2209.02778/) (2022)
- [Continuous Scene Representations for Embodied AI](https://mlsanity.com/paper/2203.17251/) (2022)
- [TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priors](https://mlsanity.com/paper/2207.10761/) (2022)
- [SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs](https://mlsanity.com/paper/2309.12188/) (2023)
- [Towards Optimal Correlational Object Search](https://mlsanity.com/paper/2110.09991/) (2021)
- [A System for Generalized 3D Multi-Object Search](https://mlsanity.com/paper/2303.03178/) (2023)
- [A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search](https://mlsanity.com/paper/2206.13396/) (2022)
- [Toward Optimal Tabletop Rearrangement with Multiple Manipulation Primitives](https://mlsanity.com/paper/2310.00167/) (2023)
- [Knowledge-Based Hierarchical POMDPs for Task Planning](https://mlsanity.com/paper/2103.10642/) (2021)
- [Task Planning for Object Rearrangement in Multi-room Environments](https://mlsanity.com/paper/2406.00451/) (2024)
- [Effective Baselines for Multiple Object Rearrangement Planning in Partially Observable Mapped Environments](https://mlsanity.com/paper/2301.09854/) (2023)

## Built on by (most cited first, 1 of 1)

- [Graph Neural Network Based Action Ranking for Planning](https://mlsanity.com/paper/2412.04752/) (2024)

## Abstract (arXiv)

We present an online planning framework and a new benchmark dataset for solving multi-object rearrangement problems in partially observable, multi-room environments. Current object rearrangement solutions, primarily based on Reinforcement Learning or hand-coded planning methods, often lack adaptability to diverse challenges. To address this limitation, we introduce a novel Hierarchical Object-Oriented Partially Observed Markov Decision Process (HOO-POMDP) planning approach. This approach comprises of (a) an object-oriented POMDP planner generating sub-goals, (b) a set of low-level policies for sub-goal achievement, and (c) an abstraction system converting the continuous low-level world into a representation suitable for abstract planning. To enable rigorous evaluation of rearrangement challenges, we introduce MultiRoomR, a comprehensive benchmark featuring diverse multi-room environments with varying degrees of partial observability (10-30% initial visibility), blocked paths, obstructed goals, and multiple objects (10-20) distributed across 2-4 rooms. Experiments demonstrate that our system effectively handles these complex scenarios while maintaining robust performance even with imperfect perception, achieving promising results across both existing benchmarks and our new MultiRoomR dataset.