# Behavior Constraining in Weight Space for Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.05479 · HTML: https://mlsanity.com/paper/2107.05479/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 7 of its 8 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)

## Builds on (most cited first, 7 of 7)

- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [Behavior Regularized Offline Reinforcement Learning](https://mlsanity.com/paper/1911.11361/) (2019)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Overcoming Model Bias for Robust Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2008.05533/) (2020)
- [A Benchmark Environment Motivated by Industrial Control Problems](https://mlsanity.com/paper/1709.09480/) (2017)
- [Particle Swarm Optimization for Generating Interpretable Fuzzy Reinforcement Learning Policies](https://mlsanity.com/paper/1610.05984/) (2016)

## Built on by (most cited first, 4 of 4)

- [User-Interactive Offline Reinforcement Learning](https://mlsanity.com/paper/2205.10629/) (2022)
- [Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learning](https://mlsanity.com/paper/2201.05433/) (2022)
- [Model-based Offline Quantum Reinforcement Learning](https://mlsanity.com/paper/2404.10017/) (2024)
- [Measuring Data Quality for Dataset Selection in Offline Reinforcement Learning](https://mlsanity.com/paper/2111.13461/) (2021)

## Abstract (arXiv)

In offline reinforcement learning, a policy needs to be learned from a single pre-collected dataset. Typically, policies are thus regularized during training to behave similarly to the data generating policy, by adding a penalty based on a divergence between action distributions of generating and trained policy. We propose a new algorithm, which constrains the policy directly in its weight space instead, and demonstrate its effectiveness in experiments.