# Scaling Internal-State Policy-Gradient Methods for POMDPs

arXiv: https://arxiv.org/abs/2512.03204 · HTML: https://mlsanity.com/paper/2512.03204/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 15 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Builds on (most cited first, 2 of 2)

- [Solving POMDPs by Searching the Space of Finite Policies](https://mlsanity.com/paper/1301.6720/) (2013)
- [Policy Improvement for POMDPs Using Normalized Importance Sampling](https://mlsanity.com/paper/1301.2310/) (2013)

## Built on by (most cited first, 1 of 1)

- [Cochain Perspectives on Temporal-Difference Signals for Learning Beyond Markov Dynamics](https://mlsanity.com/paper/2602.06939/) (2026)

## Abstract (arXiv)

Policy-gradient methods have received increased attention recently as a mechanism for learning to act in partially observable environments. They have shown promise for problems admitting memoryless policies but have been less successful when memory is required. In this paper we develop several improved algorithms for learning policies with memory in an infinite-horizon setting -- directly when a known model of the environment is available, and via simulation otherwise. We compare these algorithms on some large POMDPs, including noisy robot navigation and multi-agent problems.