# Comparison and Unification of Three Regularization Methods in Batch Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.08134 · HTML: https://mlsanity.com/paper/2109.08134/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 2 of its 7 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)

## Builds on (most cited first, 2 of 2)

- [Discount Factor as a Regularizer in Reinforcement Learning](https://mlsanity.com/paper/2007.02040/) (2020)
- [Mitigating Planner Overfitting in Model-Based Reinforcement Learning](https://mlsanity.com/paper/1812.01129/) (2018)

## Built on by (most cited first, 1 of 1)

- [On the Role of Discount Factor in Offline Reinforcement Learning](https://mlsanity.com/paper/2206.03383/) (2022)

## Abstract (arXiv)

In batch reinforcement learning, there can be poorly explored state-action pairs resulting in poorly learned, inaccurate models and poorly performing associated policies. Various regularization methods can mitigate the problem of learning overly-complex models in Markov decision processes (MDPs), however they operate in technically and intuitively distinct ways and lack a common form in which to compare them. This paper unifies three regularization methods in a common framework -- a weighted average transition matrix. Considering regularization methods in this common form illuminates how the MDP structure and the state-action pair distribution of the batch data set influence the relative performance of regularization methods. We confirm intuitions generated from the common framework by empirical evaluation across a range of MDPs and data collection policies.