mlsanityData to 7 Oct 2026

Online algorithms for POMDPs with continuous state, action, and observation spaces

arXiv 1709.06196 ↗First posted 18 Sep 2017cs.AI cs.RO cs.SYcited by 82

Not on the map. Why?

Abstract (arXiv)

Online solvers for partially observable Markov decision processes have been applied to problems with large discrete state spaces, but continuous state, action, and observation spaces remain a challenge. This paper begins by investigating double progressive widening (DPW) as a solution to this challenge. However, we prove that this modification alone is not sufficient because the belief representations in the search tree collapse to a single particle causing the algorithm to converge to a policy that is suboptimal regardless of the computation time. This paper proposes and evaluates two new algorithms, POMCPOW and PFT-DPW, that overcome this deficiency by using weighted particle filtering. Simulation results show that these modifications allow the algorithms to be successful where previous approaches fail.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 17; 2023: 19; 2024: 10; 2025: 12; 2026: 15.

82 papers.

Sort:

By area:

  1. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  2. 2022Partially Observable Markov Decision Processes in Robotics: A Surveycited by 81
  3. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
  4. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  5. 2022Optimality Guarantees for Particle Belief Approximation of POMDPscited by 20
  6. 2022Safe Output Feedback Motion Planning from Images via Learned Perception Modules and Contraction Theorycited by 18
  7. 2026VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Modelscited by 14
  8. 2023Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approachcited by 13
  9. 2023BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximationscited by 13
  10. 2021Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensingcited by 12
  11. 2024Bayes Adaptive Monte Carlo Tree Search for Offline Model-based Reinforcement Learningcited by 11
  12. 2021Compositional Learning-based Planning for Vision POMDPscited by 8
38 more
  1. 2022Epistemic Monte Carlo Tree Searchcited by 7
  2. 2023Active teacher selection for reward learningcited by 7
  3. 2024ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraintscited by 7
  4. 2021Simplified Belief-Dependent Reward MCTS Planning with Guaranteed Tree Consistencycited by 6
  5. 2022Adaptive Information Belief Space Planningcited by 6
  6. 2022A POMDP Model for Safe Geological Carbon Sequestrationcited by 6
  7. 2023Adaptive Discretization using Voronoi Trees for Continuous POMDPscited by 6
  8. 2022Mask Atari for Deep Reinforcement Learning as POMDP Benchmarkscited by 5
  9. 2022Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascentcited by 5
  10. 2024Learning Online Belief Prediction for Efficient POMDP Planning in Autonomous Drivingcited by 5
  11. 2021Monte Carlo Information-Oriented Planningcited by 4
  12. 2021Online POMDP Planning via Simplificationcited by 4
  13. 2022Intention-Aware Navigation in Crowds with Extended-Space POMDP Planningcited by 4
  14. 2022Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensingcited by 4
  15. 2023Simplified Continuous High Dimensional Belief Space Planning with Adaptive Probabilistic Belief-dependent Constraintscited by 4
  16. 2022Risk Aware Adaptive Belief-dependent Probabilistically Constrained Continuous POMDP Planningcited by 3
  17. 2022Adaptive Discretization using Voronoi Trees for Continuous-Action POMDPscited by 3
  18. 2025Managing Geological Uncertainty in Critical Mineral Supply Chains: A POMDP Approach with Application to U.S. Lithium Resourcescited by 3
  19. 2021Probabilistic Loss and its Online Characterization for Simplified Decision Making Under Uncertaintycited by 2
  20. 2021Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spacescited by 2
  21. 2021HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensingcited by 2
  22. 2022Monte Carlo Planning in Hybrid Belief POMDPscited by 2
  23. 2023Reflections from the Workshop on AI-Assisted Decision Making for Conservationcited by 2
  24. 2023Online POMDP Planning with Anytime Deterministic Optimality Guaranteescited by 2
  25. 2023Rollout Heuristics for Online Stochastic Contingent Planningcited by 2
  26. 2023Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practicecited by 2
  27. 2024Previous Knowledge Utilization In Online Anytime Belief Space Planningcited by 2
  28. 2025Observation Adaptation via Annealed Importance Resampling for Partially Observable Markov Decision Processescited by 2
  29. 2025Sequential Monte Carlo for Policy Optimization in Continuous POMDPscited by 2
  30. 2025Probabilistic Active Goal Recognitioncited by 2
  31. 2026Online Risk-Averse Planning in POMDPs Using Iterated CVaR Value Functioncited by 2
  32. 2022Risk-aware Meta-level Decision Making for Exploration Under Uncertaintycited by 1
  33. 2023Data Association Aware POMDP Planning with Hypothesis Pruning Performance Guaranteescited by 1
  34. 2023Optimizing Carbon Storage Operations for Long-Term Safetycited by 1
  35. 2023Monte-Carlo Search for an Equilibrium in Dec-POMDPscited by 1
  36. 2023Reinforcement Learning under External Influence: Guarantees, Algorithms, and Sample Complexitycited by 1
  37. 2023Measurement Simplification in ρ-POMDP with Performance Guaranteescited by 1
  38. 2023No Compromise in Solution Quality: Speeding Up Belief-dependent Continuous POMDPs via Adaptive Multilevel Simplificationcited by 1

50 most cited of 82 All 82 (JSON)