mlsanityData to 7 Oct 2026

Solving POMDPs by Searching in Policy Space

arXiv 1301.7380 ↗First posted 30 Jan 2013cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Most algorithms for solving POMDPs iteratively improve a value function that implicitly represents a policy and are said to search in value function space. This paper presents an approach to solving POMDPs that represents a policy explicitly as a finite-state controller and iteratively improves the controller by search in policy space. Two related algorithms illustrate this approach. The first is a policy iteration algorithm that can outperform value iteration in solving infinitehorizon POMDPs. It provides the foundation for a new heuristic search algorithm that promises further speedup by focusing computational effort on regions of the problem space that are reachable, or likely to be reached, from a start state.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jan 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 2; 2023: 1; 2024: 3; 2025: 2; 2026: 1.

13 papers.

Sort:

  1. 2022Learning in Observable POMDPs, without Computationally Intractable Oraclescited by 30
  2. 2022Planning in Observable POMDPs in Quasipolynomial Timecited by 25
  3. 2021Convergence of Finite Memory Q-Learning for POMDPs and Near Optimality of Learned Policies under Filter Stabilitycited by 20
  4. 2024Agent-state based policies in POMDPs: Beyond belief-state MDPscited by 9
  5. 2024Periodic agent-state based Q-learning for POMDPscited by 6
  6. 2023Welfare Maximization Algorithm for Solving Budget-Constrained Multi-Component POMDPscited by 5
  7. 2021Solving infinite-horizon Dec-POMDPs using Finite State Controllers within JESPcited by 4
  8. 2024Learning Explainable and Better Performing Representations of POMDP Strategiescited by 4
  9. 2026The Myhill-Nerode Theorem for Bounded Interaction: Canonical Abstractions via Agent-Bounded Indistinguishabilitycited by 2
  10. 2025Partially Observable Monte-Carlo Graph Searchcited by 1
  11. 2021Bayesian Nonparametric Reinforcement Learning in LTE and Wi-Fi Coexistencecited by 0
  12. 2021Bayesian Nonparametric Modelling for Model-Free Reinforcement Learning in LTE-LAA and Wi-Fi Coexistencecited by 0
1 more
  1. 2025Memoryless Policy Iteration for Episodic POMDPscited by 0