mlsanityData to 7 Oct 2026

CORL: Research-oriented Deep Offline Reinforcement Learning Library

arXiv 2210.07105 ↗First posted 13 Oct 2022cs.LG cs.AIcited by 117

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

CORL is an open-source library that provides thoroughly benchmarked single-file implementations of both deep offline and offline-to-online reinforcement learning algorithms. It emphasizes a simple developing experience with a straightforward codebase and a modern analysis tracking tool. In CORL, we isolate methods implementation into separate single files, making performance-relevant details easier to recognize. Additionally, an experiment tracking feature is available to help log metrics, hyperparameters, dependencies, and more to the cloud. Finally, we have ensured the reliability of the implementations by benchmarking commonly employed D4RL datasets providing a transparent source of results that can be reused for robust evaluation tools such as performance profiles, probability of improvement, or expected online performance.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 44 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

25 of 44 references.

Closest ancestors · 10

Same direction, newest first

  1. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters27 May
  2. 2022Supported Policy Optimization for Offline Reinforcement Learning13 Feb
  3. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  4. 2021Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matters8 Oct
  5. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  6. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning22 Sep
  7. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  8. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  10. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  4. 2017Deep Reinforcement Learning that Matterscited by 727
  5. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  6. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  7. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  8. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  9. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  10. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 58
  11. 2019ChainerRL: A Deep Reinforcement Learning Librarycited by 51
  12. 2018Horizon: Facebook's Open Source Applied Reinforcement Learning Platformcited by 48
3 more
  1. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  2. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planningcited by 23
  3. 2019Catalyst.RL: A Distributed Framework for Reproducible RL Researchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 41; 2025: 34; 2026: 18.

117 papers.

Sort:

By area:

  1. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  2. 2025Flow Q-Learningcited by 162
  3. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 121
  4. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  5. 2023Synthetic Experience Replaycited by 100
  6. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 82
  7. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 65
  8. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  9. 2025Horizon Reduction Makes RL Scalablecited by 55
  10. 2023Anti-Exploration by Random Network Distillationcited by 49
  11. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  12. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitchingcited by 35
38 more
  1. 2024Diffusion Model Predictive Controlcited by 35
  2. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learningcited by 33
  3. 2024Exploration and Anti-Exploration with Distributional Random Network Distillationcited by 32
  4. 2023Corruption-Robust Offline Reinforcement Learning with General Function Approximationcited by 30
  5. 2025Scaling Offline RL via Efficient and Expressive Shortcut Modelscited by 27
  6. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 26
  7. 2024Reinformer: Max-Return Sequence Modeling for Offline RLcited by 25
  8. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  9. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 24
  10. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
  11. 2023Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learningcited by 24
  12. 2023Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimizationcited by 23
  13. 2024GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learningcited by 22
  14. 2024The Prevalence of Neural Collapse in Neural Multivariate Regressioncited by 21
  15. 2023ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensemblescited by 20
  16. 2024Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedbackcited by 20
  17. 2024AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimizationcited by 19
  18. 2024Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learningcited by 17
  19. 2025One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlowcited by 16
  20. 2023IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Controlcited by 14
  21. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learningcited by 14
  22. 2024Listwise Reward Estimation for Offline Preference-based Reinforcement Learningcited by 14
  23. 2024Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learningcited by 13
  24. 2023Balancing policy constraint and ensemble size in uncertainty-based offline reinforcement learningcited by 12
  25. 2024Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Plannercited by 12
  26. 2025Flow-Based Single-Step Completion for Efficient and Expressive Policy Learningcited by 12
  27. 2024Unsupervised-to-Online Reinforcement Learningcited by 11
  28. 2023Improving Offline RL by Blending Heuristicscited by 10
  29. 2023Iteratively Refined Behavior Regularization for Offline Reinforcement Learningcited by 10
  30. 2025A Clean Slate for Offline Reinforcement Learningcited by 10
  31. 2024Optimistic Critic Reconstruction and Constrained Fine-Tuning for General Offline-to-Online RLcited by 9
  32. 2024The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learningcited by 8
  33. 2024Rethinking Optimal Transport in Offline Reinforcement Learningcited by 8
  34. 2025Vintix: Action Model via In-Context Reinforcement Learningcited by 8
  35. 2025Yes, Q-learning Helps Offline In-Context RLcited by 8
  36. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learningcited by 7
  37. 2025Intention-Conditioned Flow Occupancy Modelscited by 7
  38. 2025Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Datacited by 7

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning
  2. 2025NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios
  3. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
  4. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  5. 2023Improving and Benchmarking Offline Reinforcement Learning Algorithms
  6. 2021d3rlpy: An Offline Deep Reinforcement Learning Library
  7. 2024AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset
  8. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothing