Model approximation in MDPs with unbounded per-step cost
Not on the map. Why?
Abstract (arXiv)
We consider the problem of designing a control policy for an infinite-horizon discounted cost Markov decision process M when we only have access to an approximate model hatM. How well does an optimal policy hatπstar of the approximate model perform when used in the original model M? We answer this question by bounding a weighted norm of the difference between the value function of hatπ^star when used in M and the optimal value function of M. We then extend our results and obtain potentially tighter upper bounds by considering affine transformations of the per-step cost. We further provide upper bounds that explicitly depend on the weighted distance between cost functions and weighted distance between transition kernels of the original and approximate models. We present examples to illustrate our results.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 10 of its 45 references that are in the map. First posted on arXiv 13 Feb 2024.
Builds on
10 of 45 references.
References · 10
- 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
- 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 67
- 2017Near Optimal Behavior via Approximate State Abstractioncited by 63
- 2020Plannable Approximations to MDP Homomorphisms: Equivariance under Actionscited by 61
- 2020Near Optimality of Finite Memory Feedback Policies in Partially Observed Markov Decision Processescited by 20
- 2017On overfitting and asymptotic bias in batch reinforcement learning with partial observabilitycited by 16
- 2015Asymptotic Optimality of Finite Approximations to Markov Decision Processes with Borel Spacescited by 11
- 2018Robustness to incorrect system models in stochastic controlcited by 5
- 2014Near Optimality of Quantized Policies in Stochastic Control Under Weak Continuity Conditionscited by 4
- 2018Robustness to incorrect priors in partially observed stochastic controlcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.