Reinforcement Learning in POMDP's via Direct Gradient Ascent
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
This paper discusses theoretical and experimental aspects of gradient-based approaches to the direct optimization of policy performance in controlled POMDPs. We introduce GPOMDP, a REINFORCE-like algorithm for estimating an approximation to the gradient of the average reward as a function of the parameters of a stochastic policy. The algorithm's chief advantages are that it requires only a single sample path of the underlying Markov chain, it uses only one free parameter β∈ [0,1), which has a natural interpretation in terms of bias-variance trade-off, and it requires no knowledge of the underlying state. We prove convergence of GPOMDP and show how the gradient estimates produced by GPOMDP can be used in a conjugate-gradient procedure to find local optima of the average reward.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 0 of its 34 references that are in the map. First posted on arXiv 2 Dec 2025.
Builds on
0 of 34 references.
No references here yet.
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025Memoryless Policy Iteration for Episodic POMDPscited by 0
- 2025Memoryless Policy Iteration for Episodic POMDPs11 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2026Policy Gradient Methods for Non-Markovian Reinforcement Learning
- 2024A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms
- 2025A Two-Timescale Primal-Dual Framework for Reinforcement Learning via Online Dual Variable Guidance
- 2025Policy Gradient with Tree Search: Avoiding Local Optimas through Lookahead
- 2024Sample-Efficient Constrained Reinforcement Learning with General Parameterization
- 2025Policy Gradient with Second Order Momentum
- 2026Mathematical methods of reinforcement learning
- 2024Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization