mlsanityData to 13 Oct 2026

Critic Regularized Regression

arXiv 2006.15134 ↗First posted 26 Jun 2020cs.LG cs.AI stat.MLcited by 270

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline reinforcement learning (RL), also known as batch RL, offers the prospect of policy optimization from large pre-recorded datasets without online environment interaction. It addresses challenges with regard to the cost of data collection and safety, both of which are particularly pertinent to real-world applications of RL. Unfortunately, most off-policy algorithms perform poorly when learning from a fixed dataset. In this paper, we propose a novel offline RL algorithm to learn policies from data using a form of critic-regularized regression (CRR). We find that CRR performs surprisingly well and scales to tasks with high-dimensional state and action spaces -- outperforming several state-of-the-art offline RL algorithms by a significant margin on a wide range of benchmark tasks.

Key facts. Cited by 270 papers in the map (data through 13 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 26 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2020: 7; 2021: 40; 2022: 60; 2023: 62; 2024: 44; 2025: 29; 2026: 28.

270 papers.

Sort:

By area:

  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,257
  2. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 884
  3. 2022A Generalist Agentcited by 803
  4. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 704
  5. 2020Is Pessimism Provably Efficient for Offline RL?cited by 363
  6. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 314
  7. 2025π*0.6: a VLA That Learns From Experiencecited by 286
  8. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 277
  9. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  10. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 211
  11. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  12. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
38 more
  1. 2021Offline RL Without Off-Policy Evaluationcited by 150
  2. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  3. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 138
  4. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 137
  5. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
  6. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 130
  7. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  8. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  9. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularizationcited by 115
  10. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 114
  11. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learningcited by 112
  12. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
  13. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  14. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 103
  15. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 94
  16. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  17. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  18. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  19. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 87
  20. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  21. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 84
  22. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 83
  23. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 76
  24. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 75
  25. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 74
  26. 2022Supported Policy Optimization for Offline Reinforcement Learningcited by 73
  27. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  28. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  29. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 70
  30. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 66
  31. 2023Elastic Decision Transformercited by 66
  32. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  33. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 61
  34. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 58
  35. 2025Horizon Reduction Makes RL Scalablecited by 56
  36. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 52
  37. 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50
  38. 2025The Best Instruction-Tuning Data are Those That Fitcited by 50

50 most cited of 270 All 270 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning
  3. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learning
  4. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
  5. 2023A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
  6. 2021Representation Matters: Offline Pretraining for Sequential Decision Making
  7. 2025Fewer May Be Better: Enhancing Offline Reinforcement Learning with Reduced Dataset
  8. 2020MOPO: Model-based Offline Policy Optimization