mlsanityData to 7 Oct 2026

Partially Observable Markov Decision Processes (POMDPs) and Robotics

arXiv 2107.07599 ↗First posted 15 Jul 2021cs.ROcited by 49

Not on the map. Why?

Abstract (arXiv)

Planning under uncertainty is critical to robotics. The Partially Observable Markov Decision Process (POMDP) is a mathematical framework for such planning problems. It is powerful due to its careful quantification of the non-deterministic effects of actions and partial observability of the states. But precisely because of this, POMDP is notorious for its high computational complexity and deemed impractical for robotics. However, since early 2000, POMDPs solving capabilities have advanced tremendously, thanks to sampling-based approximate solvers. Although these solvers do not generate the optimal solution, they can compute good POMDP solutions that significantly improve the robustness of robotics systems within reasonable computational resources, thereby making POMDPs practical for many realistic robotics problems. This paper presents a review of POMDPs, emphasizing computational issues that have hindered its practicality in robotics and ideas in sampling-based solvers that have alleviated such difficulties, together with lessons learned from applying POMDPs to physical robots.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jul 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 14; 2024: 8; 2025: 14; 2026: 6.

49 papers.

Sort:

By area:

  1. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  2. 2023Lemur: Harmonizing Natural Language and Code for Language Agentscited by 71
  3. 2022Visuo-Tactile Transformers for Manipulationcited by 43
  4. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  5. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  6. 2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 17
  7. 2021Towards Optimal Correlational Object Searchcited by 14
  8. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  9. 2023CAR-DESPOT: Causally-Informed Online POMDP Planning for Robots in Confounded Environmentscited by 11
  10. 2023Digital Twin-Enhanced Wireless Indoor Navigation: Achieving Efficient Environment Sensing with Zero-Shot Reinforcement Learningcited by 11
  11. 2022Active Inference for Autonomous Decision-Making with Contextual Multi-Armed Banditscited by 6
  12. 2023Adaptive Discretization using Voronoi Trees for Continuous POMDPscited by 6
37 more
  1. 2024Believing is Seeing: Unobserved Object Detection using Generative Modelscited by 4
  2. 2022Adaptive Discretization using Voronoi Trees for Continuous-Action POMDPscited by 3
  3. 2023Towards Probabilistic Causal Discovery, Inference & Explanations for Autonomous Drones in Mine Surveying Taskscited by 3
  4. 2023Observation-Augmented Contextual Multi-Armed Bandits for Robotic Search and Explorationcited by 3
  5. 2026TopoCurate:Modeling Interaction Topology for Tool-Use Agent Trainingcited by 3
  6. 2023Rollout Heuristics for Online Stochastic Contingent Planningcited by 2
  7. 2024The State of Robot Motion Generationcited by 2
  8. 2024Learning to Look: Seeking Information for Decision Making via Policy Factorizationcited by 2
  9. 2025Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlationscited by 2
  10. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  11. 2025Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learningcited by 2
  12. 2023Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environmentscited by 1
  13. 2023Robust Recurrent Reinforcement Learning under Evolving Hidden Disturbances with Application to Rover Wheel Slipcited by 1
  14. 2024Addressing Myopic Constrained POMDP Planning with Recursive Dual Ascentcited by 1
  15. 2024Preparing for Black Swans: The Antifragility Imperative for Machine Learningcited by 1
  16. 2024Privacy Preserving Reinforcement Learning for Population Processescited by 1
  17. 2024Design of Restricted Normalizing Flow towards Arbitrary Stochastic Policy with Computational Efficiencycited by 1
  18. 2025Partially Observable Monte-Carlo Graph Searchcited by 1
  19. 2025MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Modelscited by 1
  20. 2025ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPscited by 1
  21. 2026Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learningcited by 1
  22. 2026Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learningcited by 1
  23. 2022Explicit User Manipulation in Reinforcement Learning Based Recommender Systemscited by 0
  24. 2023A Surprisingly Simple Continuous-Action POMDP Solver: Lazy Cross-Entropy Search Over Policy Treescited by 0
  25. 2023Value of Assistance for Graspingcited by 0
  26. 2023An Intelligent Social Learning-based Optimization Strategy for Black-box Robotic Control with Reinforcement Learningcited by 0
  27. 2023Markov Decision Processes with Noisy State Observationcited by 0
  28. 2024COBRA-PPM: A Causal Bayesian Reasoning Architecture Using Probabilistic Programming for Robot Manipulation Under Uncertaintycited by 0
  29. 2025Multi-source Plume Tracing via Multi-Agent Reinforcement Learningcited by 0
  30. 2025Bi-directional Recurrence Improves Transformer in Partially Observable Markov Decision Processescited by 0
  31. 2025Control Synthesis in Partially Observable Environments for Complex Perception-Related Objectivescited by 0
  32. 2025Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisationcited by 0
  33. 2025Shared Control of Holonomic Wheelchairs through Reinforcement Learningcited by 0
  34. 2025Neural Value Iterationcited by 0
  35. 2026MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidancecited by 0
  36. 2026Finite-Time Analysis of MCTS in Continuous POMDP Planningcited by 0
  37. 2026Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learningcited by 0