On shallow planning under partial observability
Not on the map. Why?
Abstract (arXiv)
Formulating a real-world problem under the Reinforcement Learning framework involves non-trivial design choices, such as selecting a discount factor for the learning objective (discounted cumulative rewards), which articulates the planning horizon of the agent. This work investigates the impact of the discount factor on the bias-variance trade-off given structural parameters of the underlying Markov Decision Process. Our results support the idea that a shorter planning horizon might be beneficial, especially under partial observability.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 22 Jul 2024.
Builds on
18 of 33 references.
References · 18
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
- 2022When Is Partially Observable Reinforcement Learning Not Scary?cited by 100
- 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 75
- 2017Near Optimal Behavior via Approximate State Abstractioncited by 63
- 2020Discount Factor as a Regularizer in Reinforcement Learningcited by 44
- 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
- 2019Dealing with Sparse Rewards in Reinforcement Learningcited by 40
- 2019Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approachcited by 32
- 2023Bridging RL Theory and Practice with the Effective Horizoncited by 31
6 more
- 2015How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategiescited by 26
- 2017On overfitting and asymptotic bias in batch reinforcement learning with partial observabilitycited by 16
- 2021Uniform-PAC Bounds for Reinforcement Learning with Linear Function Approximationcited by 14
- 2023Reducing Blackwell and Average Optimality to Discounted MDPs via the Blackwell Discount Factorcited by 13
- 2021The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPscited by 9
- 2023Uniform-PAC Guarantees for Model-Based RL with Bounded Eluder Dimensioncited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.