# Policy Error Bounds for Model-Based Reinforcement Learning with Factored Linear Models

arXiv: https://arxiv.org/abs/1602.06346 · HTML: https://mlsanity.com/paper/1602.06346/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2016.

## Built on by (most cited first, 5 of 5)

- [Operator Splitting Value Iteration](https://mlsanity.com/paper/2211.13937/) (2022)
- [Maximum Entropy Model Correction in Reinforcement Learning](https://mlsanity.com/paper/2311.17855/) (2023)
- [UVIP: Model-Free Approach to Evaluate Reinforcement Learning Algorithms](https://mlsanity.com/paper/2105.02135/) (2021)
- [Primal-dual regression approach for Markov decision processes with general state and action space](https://mlsanity.com/paper/2210.00258/) (2022)
- [On Task-Relevant Loss Functions in Meta-Reinforcement Learning and Online LQR](https://mlsanity.com/paper/2312.05465/) (2023)

## Abstract (arXiv)

In this paper we study a model-based approach to calculating approximately optimal policies in Markovian Decision Processes. In particular, we derive novel bounds on the loss of using a policy derived from a factored linear model, a class of models which generalize numerous previous models out of those that come with strong computational guarantees. For the first time in the literature, we derive performance bounds for model-based techniques where the model inaccuracy is measured in weighted norms. Moreover, our bounds show a decreased sensitivity to the discount factor and, unlike similar bounds derived for other approaches, they are insensitive to measure mismatch. Similarly to previous works, our proofs are also based on contraction arguments, but with the main differences that we use carefully constructed norms building on Banach lattices, and the contraction property is only assumed for operators acting on "compressed" spaces, thus weakening previous assumptions, while strengthening previous results.