# Robotic manipulation of multiple objects as a POMDP

arXiv: https://arxiv.org/abs/1402.0649 · HTML: https://mlsanity.com/paper/1402.0649/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2014.

## Built on by (most cited first, 9 of 9)

- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Memory-based Deep Reinforcement Learning for POMDPs](https://mlsanity.com/paper/2102.12344/) (2021)
- [LLM-Guided Probabilistic Program Induction for POMDP Model Estimation](https://mlsanity.com/paper/2505.02216/) (2025)
- [ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints](https://mlsanity.com/paper/2405.00644/) (2024)
- [Knowledge-Based Hierarchical POMDPs for Task Planning](https://mlsanity.com/paper/2103.10642/) (2021)
- [Reinforcement Learning using Guided Observability](https://mlsanity.com/paper/2104.10986/) (2021)
- [Distributed Mission Planning of Complex Tasks for Heterogeneous Multi-Robot Teams](https://mlsanity.com/paper/2109.10106/) (2021)
- [Neuro-Symbolic Manipulation Understanding with Enriched Semantic Event Chains](https://mlsanity.com/paper/2604.21053/) (2026)
- [Linear programming-based solution methods for constrained partially observable Markov decision processes](https://mlsanity.com/paper/2206.14081/) (2022)

## Abstract (arXiv)

This paper investigates manipulation of multiple unknown objects in a crowded environment. Because of incomplete knowledge due to unknown objects and occlusions in visual observations, object observations are imperfect and action success is uncertain, making planning challenging. We model the problem as a partially observable Markov decision process (POMDP), which allows a general reward based optimization objective and takes uncertainty in temporal evolution and partial observations into account. In addition to occlusion dependent observation and action success probabilities, our POMDP model also automatically adapts object specific action success probabilities. To cope with the changing system dynamics and performance constraints, we present a new online POMDP method based on particle filtering that produces compact policies. The approach is validated both in simulation and in physical experiments in a scenario of moving dirty dishes into a dishwasher. The results indicate that: 1) a greedy heuristic manipulation approach is not sufficient, multi-object manipulation requires multi-step POMDP planning, and 2) on-line planning is beneficial since it allows the adaptation of the system dynamics model based on actual experience.