Learning POMDPs with Linear Function Approximation and Finite Memory
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
We study reinforcement learning with linear function approximation and finite-memory approximations for partially observed Markov decision processes (POMDPs). We first present an algorithm for the value evaluation of finite-memory feedback policies. We provide error bounds derived from filter stability and projection errors. We then study the learning of finite-memory based near-optimal Q values. Convergence in this case requires further assumptions on the exploration policy when using general basis functions. We then show that these assumptions can be relaxed for specific models such as those with perfectly linear cost and dynamics, or when using discretization based basis functions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 20 references that are in the map. First posted on arXiv 20 May 2025.
Builds on
7 of 20 references.
References · 7
- 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
- 2020Near Optimality of Finite Memory Feedback Policies in Partially Observed Markov Decision Processescited by 23
- 2021Convergence of Finite Memory Q-Learning for POMDPs and Near Optimality of Learned Policies under Filter Stabilitycited by 22
- 2022Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiencycited by 16
- 2021Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuitycited by 11
- 2018Weak Feller Property of Non-linear Filterscited by 6
- 2024A Functional Model Method for Nonconvex Nonsmooth Conditional Stochastic Optimizationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reinforcement Learning with Function Approximation for Non-Markov Processes
- 2024Near Optimal Approximations and Finite Memory Policies for POMPDs with Continuous Spaces ↗
- 2025Scalable Policy-Based RL Algorithms for POMDPs
- 2024Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning
- 2024Refined Bounds on Near Optimality Finite Window Policies in POMDPs and Their Reinforcement Learning
- 2025Addressing Finite-Horizon MDPs via Low-Rank Tensor Value Approximation
- 2025Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modeling
- 2023Safe Policy Improvement for POMDPs via Finite-State Controllers