mlsanityData to 7 Oct 2026

Behavior Constraining in Weight Space for Offline Reinforcement Learning

arXiv 2107.05479 ↗First posted 12 Jul 2021cs.LGcited by 4

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

In offline reinforcement learning, a policy needs to be learned from a single pre-collected dataset. Typically, policies are thus regularized during training to behave similarly to the data generating policy, by adding a penalty based on a divergence between action distributions of generating and trained policy. We propose a new algorithm, which constrains the policy directly in its weight space instead, and demonstrate its effectiveness in experiments.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 7 of its 8 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

7 of 8 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Overcoming Model Bias for Robust Offline Deep Reinforcement Learning12 Aug
  2. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 5

  1. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  2. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  3. 2017Deep Reinforcement Learning: An Overviewcited by 327
  4. 2017A Benchmark Environment Motivated by Industrial Control Problemscited by 31
  5. 2016Particle Swarm Optimization for Generating Interpretable Fuzzy Reinforcement Learning Policiescited by 19

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 0; 2024: 1.

4 papers.

  1. 2022User-Interactive Offline Reinforcement Learningcited by 11
  2. 2022Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learningcited by 9
  3. 2024Model-based Offline Quantum Reinforcement Learningcited by 4
  4. 2021Measuring Data Quality for Dataset Selection in Offline Reinforcement Learningcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learning
  2. 2023Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
  3. 2024SelfBC: Self Behavior Cloning for Offline Reinforcement Learning
  4. 2023Goal-conditioned Offline Reinforcement Learning through State Space Partitioning
  5. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
  6. 2023Iteratively Refined Behavior Regularization for Offline Reinforcement Learning
  7. 2025Adaptive Scaling of Policy Constraints for Offline Reinforcement Learning
  8. 2025Policy Constraint by Only Support Constraint for Offline Reinforcement Learning