# Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks

arXiv: https://arxiv.org/abs/1809.01906 · HTML: https://mlsanity.com/paper/1809.01906/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 3 of 3)

- [A novel policy for pre-trained Deep Reinforcement Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2101.00738/) (2021)
- [Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow](https://mlsanity.com/paper/2103.14407/) (2021)
- [Variational Inference for Model-Free and Model-Based Reinforcement Learning](https://mlsanity.com/paper/2209.01693/) (2022)

## Abstract (arXiv)

This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks (DQNs) by adding a model-learning component yielding a transcoder network. The prediction errors for the model are included in the basic DQN loss as additional regularizers. This augmented objective leads to a richer training signal that provides feedback at every time step. Moreover, because learning an environment model shares a common structure with the RL problem, we hypothesize that the resulting objective improves both sample efficiency and performance. We empirically confirm our hypothesis on a range of 20 games from the Atari benchmark attaining superior results over vanilla DQN without model-based regularization.