# Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness

arXiv: https://arxiv.org/abs/2403.10454 · HTML: https://mlsanity.com/paper/2403.10454/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 49 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 23 of 23)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?](https://mlsanity.com/paper/2006.14911/) (2020)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [Learning Symbolic Operators for Task and Motion Planning](https://mlsanity.com/paper/2103.00589/) (2021)
- [Predicate Invention for Bilevel Planning](https://mlsanity.com/paper/2203.09634/) (2022)
- [Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models](https://mlsanity.com/paper/2206.10787/) (2022)
- [Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances](https://mlsanity.com/paper/2108.04145/) (2021)
- [Bayesian Robust Optimization for Imitation Learning](https://mlsanity.com/paper/2007.12315/) (2020)
- [3DP3: 3D Scene Perception via Probabilistic Programming](https://mlsanity.com/paper/2111.00312/) (2021)
- [Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning](https://mlsanity.com/paper/2211.01576/) (2022)
- [Discovering State and Action Abstractions for Generalized Task and Motion Planning](https://mlsanity.com/paper/2109.11082/) (2021)
- [Learning Efficient Abstract Planning Models that Choose What to Predict](https://mlsanity.com/paper/2208.07737/) (2022)
- [Motions in Microseconds via Vectorized Sampling-Based Planning](https://mlsanity.com/paper/2309.14545/) (2023)
- [Anytime Integrated Task and Motion Policies for Stochastic Environments](https://mlsanity.com/paper/1904.13006/) (2019)
- [Task-Directed Exploration in Continuous POMDPs for Robotic Manipulation of Articulated Objects](https://mlsanity.com/paper/2212.04554/) (2022)
- [3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation](https://mlsanity.com/paper/2302.03744/) (2023)
- [Bayes3D: fast learning and inference in structured generative models of 3D objects and scenes](https://mlsanity.com/paper/2312.08715/) (2023)
- [Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving](https://mlsanity.com/paper/2106.12194/) (2021)
- [Symbolic Search for Optimal Planning with Expressive Extensions](https://mlsanity.com/paper/2204.00288/) (2022)
- [Robust-RRT: Probabilistically-Complete Motion Planning for Uncertain Nonlinear Systems](https://mlsanity.com/paper/2205.07728/) (2022)

## Built on by (most cited first, 17 of 17)

- [QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?](https://mlsanity.com/paper/2503.22674/) (2025)
- [VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning](https://mlsanity.com/paper/2410.23156/) (2024)
- [PoE-World: Compositional World Modeling with Products of Programmatic Experts](https://mlsanity.com/paper/2505.10819/) (2025)
- [Evaluating Uncertainty and Quality of Vision-Language-Action-enabled Robots](https://mlsanity.com/paper/2507.17049/) (2025)
- [LLM-Guided Probabilistic Program Induction for POMDP Model Estimation](https://mlsanity.com/paper/2505.02216/) (2025)
- [Flow-based Domain Randomization for Learning and Sequencing Robotic Skills](https://mlsanity.com/paper/2502.01800/) (2025)
- [The State of Robot Motion Generation](https://mlsanity.com/paper/2410.12172/) (2024)
- [GUIDEd Agents: Enhancing Navigation Policies through Task-Specific Uncertainty Abstraction in Localization-Limited Environments](https://mlsanity.com/paper/2410.15178/) (2024)
- [Enhancing Robot Navigation Policies with Task-Specific Uncertainty Managements](https://mlsanity.com/paper/2505.13837/) (2025)
- [Large-Language-Model-Guided State Estimation for Partially Observable Task and Motion Planning](https://mlsanity.com/paper/2603.03704/) (2026)
- [ILeSiA: Interactive Learning of Robot Situational Awareness from Camera Input](https://mlsanity.com/paper/2409.20173/) (2024)
- [Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information](https://mlsanity.com/paper/2508.11252/) (2025)
- [Joint Learning of Hierarchical Neural Options and Abstract World Model](https://mlsanity.com/paper/2602.02799/) (2026)
- [GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning](https://mlsanity.com/paper/2609.19315/) (2026)
- [Task and Skill Planning: Hierarchical Robot Planning with Black-Box Skills](https://mlsanity.com/paper/2504.17901/) (2025)
- [FlatLands: Generative Floormap Completion From a Single Egocentric View](https://mlsanity.com/paper/2603.16016/) (2026)
- [Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planning](https://mlsanity.com/paper/2605.15975/) (2026)

## Abstract (arXiv)

Integrated task and motion planning (TAMP) has proven to be a valuable approach to generalizable long-horizon robotic manipulation and navigation problems. However, the typical TAMP problem formulation assumes full observability and deterministic action effects. These assumptions limit the ability of the planner to gather information and make decisions that are risk-aware. We propose a strategy for TAMP with Uncertainty and Risk Awareness (TAMPURA) that is capable of efficiently solving long-horizon planning problems with initial-state and action outcome uncertainty, including problems that require information gathering and avoiding undesirable and irreversible outcomes. Our planner reasons under uncertainty at both the abstract task level and continuous controller level. Given a set of closed-loop goal-conditioned controllers operating in the primitive action space and a description of their preconditions and potential capabilities, we learn a high-level abstraction that can be solved efficiently and then refined to continuous actions for execution. We demonstrate our approach on several robotics problems where uncertainty is a crucial factor and show that reasoning under uncertainty in these problems outperforms previously proposed determinized planning, direct search, and reinforcement learning strategies. Lastly, we demonstrate our planner on two real-world robotics problems using recent advancements in probabilistic perception.