mlsanityData to 7 Oct 2026

Partially Observable Markov Decision Processes in Robotics: A Survey

arXiv 2209.10342 ↗First posted 21 Sep 2022cs.RO cs.AI cs.SYcited by 81

Not on the map. Why?

Abstract (arXiv)

Noisy sensing, imperfect control, and environment changes are defining characteristics of many real-world robot tasks. The partially observable Markov decision process (POMDP) provides a principled mathematical framework for modeling and solving robot decision and control tasks under uncertainty. Over the last decade, it has seen many successful applications, spanning localization and navigation, search and tracking, autonomous driving, multi-robot systems, manipulation, and human-robot interaction. This survey aims to bridge the gap between the development of POMDP models and algorithms at one end and application to diverse robot decision tasks at the other. It analyzes the characteristics of these tasks and connects them with the mathematical and algorithmic properties of the POMDP framework for effective modeling and solution. For practitioners, the survey provides some of the key task characteristics in deciding when and how to apply POMDPs to robot tasks successfully. For POMDP algorithm designers, the survey provides new insights into the unique challenges of applying POMDPs to robot systems and points to promising new directions for further research.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Builds on 38 of its 180 references that are in the map. First posted on arXiv 21 Sep 2022.

Builds on

38 of 180 references.

References · 38

  1. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  4. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  5. 2016DESPOT: Online POMDP Planning with Regularizationcited by 105
  6. 2013Data-Driven Grasp Synthesis - A Surveycited by 104
  7. 2017Online algorithms for POMDPs with continuous state, action, and observation spacescited by 82
  8. 2017Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learningcited by 81
  9. 2019Online Replanning in Belief Space for Partially Observable Task and Motion Problemscited by 60
  10. 2014Online Planning Algorithms for POMDPscited by 42
  11. 2018PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrianscited by 36
  12. 2013Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processescited by 24
26 more
  1. 2019Differentiable Algorithm Networks for Composable Robot Learningcited by 24
  2. 2018HyP-DESPOT: A Hybrid Parallel Algorithm for Online Planning under Uncertaintycited by 18
  3. 2020MAGIC: Learning Macro-Actions for Online POMDP Planningcited by 17
  4. 2017The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Drivingcited by 15
  5. 2020Multiagent Rollout and Policy Iteration for POMDP with Application to Multi-Robot Repair Problemscited by 13
  6. 2016POMDP-lite for Robust Robot Planning under Uncertaintycited by 12
  7. 2019Learning Multi-Robot Decentralized Macro-Action-Based Policies via a Centralized Q-Netcited by 12
  8. 2021PLGRIM: Hierarchical Value Learning for Large-scale Exploration in Unknown Environmentscited by 12
  9. 2015Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actionscited by 11
  10. 2019Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functionscited by 11
  11. 2014Robotic manipulation of multiple objects as a POMDPcited by 9
  12. 2020Belief-Grounded Networks for Accelerated Robot Learning under Partial Observabilitycited by 9
  13. 2020Multi-Resolution POMDP Planning for Multi-Object Search in 3Dcited by 8
  14. 2020Exploiting Submodular Value Functions For Scaling Up Active Perceptioncited by 6
  15. 2022FIG-OP: Exploring Large-Scale Unknown Environments on a Fixed Time Budgetcited by 6
  16. 2015Planning for robotic exploration based on forward simulationcited by 5
  17. 2018Optimal Continuous State POMDP Planning with Semantic Observations: A Variational Approachcited by 5
  18. 2020POMP: Pomcp-based Online Motion Planning for active visual search in indoor environmentscited by 5
  19. 2020Multi-agent active perception with prediction rewardscited by 4
  20. 2017Multi-Robot Active Information Gathering with Periodic Communicationcited by 3
  21. 2018A Decision-theoretic Approach to Detection-based Target Search with a UAVcited by 3
  22. 2018Bounded Policy Synthesis for POMDPs with Safe-Reachability Objectivescited by 3
  23. 2018Estimation and Control Using Sampling-Based Bayesian Reinforcement Learningcited by 3
  24. 2018POMDP Model Learning for Human Robot Collaborationcited by 2
  25. 2016Optimizing Gaze Direction in a Visual Navigation Taskcited by 1
  26. 2018Autonomous Vehicle Speed Control for Safe Navigation of Occluded Pedestrian Crosswalkcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 16; 2025: 25; 2026: 33.

81 papers.

Sort:

By area:

  1. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  2. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  3. 2026Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learningcited by 28
  4. 2025AVA-VLA: Improving Vision-Language-Action models with Active Visual Attentioncited by 22
  5. 2024emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimationcited by 20
  6. 2025A Survey on Active Feature Acquisition Strategiescited by 16
  7. 2023BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximationscited by 13
  8. 2024Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Ratecited by 10
  9. 2024Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learningcited by 10
  10. 2023The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Modelscited by 9
  11. 2023Minimax Optimal Q Learning with Nearest Neighborscited by 8
  12. 2023Safe POMDP Online Planning via Shieldingcited by 8
38 more
  1. 2024ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraintscited by 7
  2. 2025AFABench: A Generic Framework for Benchmarking Active Feature Acquisitioncited by 7
  3. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  4. 2026PABU: Progress-Aware Belief Update for Efficient LLM Agentscited by 6
  5. 2023Fairness and Sequential Decision Making: Limits, Lessons, and Opportunitiescited by 5
  6. 2026A Brain-inspired Embodied Intelligence for Fluid and Fast Reflexive Robotics Controlcited by 5
  7. 2026DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Taskscited by 5
  8. 2024ComTraQ-MPC: Meta-Trained DQN-MPC Integration for Trajectory Tracking with Limited Active Localization Updatescited by 4
  9. 2025Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigationcited by 4
  10. 2025Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samplescited by 4
  11. 2026NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulationcited by 4
  12. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  13. 2025Improved Robustness of Deep Reinforcement Learning for Control of Time-Varying Systems by Bounded Extremum Seekingcited by 3
  14. 2023Recursively-Constrained Partially Observable Markov Decision Processescited by 2
  15. 2024Mixed-Initiative Human-Robot Teaming under Suboptimality with Online Bayesian Adaptationcited by 2
  16. 2024GASE: Graph Attention Sampling with Edges Fusion for Solving Vehicle Routing Problemscited by 2
  17. 2024Sound Heuristic Search Value Iteration for Undiscounted POMDPs with Reachability Objectivescited by 2
  18. 2024Learning to Look: Seeking Information for Decision Making via Policy Factorizationcited by 2
  19. 2024TAB-Fields: A Maximum Entropy Framework for Mission-Aware Adversarial Planningcited by 2
  20. 2025NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarioscited by 2
  21. 2025Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlationscited by 2
  22. 2025Data-Efficient Safe Policy Improvement Using Parametric Structurecited by 2
  23. 2025Graph-Enhanced Policy Optimization in LLM Agent Trainingcited by 2
  24. 2025Vectorized Online POMDP Planningcited by 2
  25. 2025PRISM: A Personality-Driven Multi-Agent Framework for Social Media Simulationcited by 2
  26. 2026Identifying Adversary Tactics and Techniques in Malware Binaries with an LLM Agentcited by 2
  27. 2026Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shiftcited by 2
  28. 20263D-Anchored Lookahead Planning for Persistent Robotic Scene Memory via World-Model-Based MCTScited by 2
  29. 2024Entropy-regularized Point-based Value Iterationcited by 1
  30. 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Modelscited by 1
  31. 2024Context-Based Echo State Networks with Prediction Confidence for Human-Robot Shared Controlcited by 1
  32. 2025Quantifying First-Order Markov Violations in Noisy Reinforcement Learning: A Causal Discovery Approachcited by 1
  33. 2025Partially Observable Monte-Carlo Graph Searchcited by 1
  34. 2025ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPscited by 1
  35. 2026Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learningcited by 1
  36. 2026Distributed Online Convex Optimization with Nonseparable Costs and Constraintscited by 1
  37. 2026A Covering Framework for Offline POMDPs Learning using Belief Space Metriccited by 1
  38. 2026Model-Based Learning of Near-Optimal Finite-Window Policies in POMDPscited by 1

50 most cited of 81 All 81 (JSON)