# Offline RL algorithms

Area: [Offline RL and imitation learning from data](https://mlsanity.com/offlinerl/index.md). HTML: https://mlsanity.com/offlinerl/offline-rl-algorithms/

## Key facts

- Offline RL algorithms: 819 papers on arXiv since 2018 (data through 7 Oct 2026).
- 440 of its 819 mapped papers (54%) cite “Conservative Q-Learning for Offline Reinforcement Learning” (2020, arXiv 2006.04779).
- 416 of its 819 mapped papers (51%) cite “Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems” (2020, arXiv 2005.01643).
- 10 new papers 17 Sep–7 Oct.

## Most cited here

- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020), cited by 440 papers here
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020), cited by 416 papers here
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020), cited by 391 papers here
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021), cited by 310 papers here
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021), cited by 302 papers here
- [Behavior Regularized Offline Reinforcement Learning](https://mlsanity.com/paper/1911.11361/) (2019), cited by 256 papers here
- [MOReL : Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2005.05951/) (2020), cited by 216 papers here
- [MOPO: Model-based Offline Policy Optimization](https://mlsanity.com/paper/2005.13239/) (2020), cited by 213 papers here
- [COMBO: Conservative Offline Model-Based Policy Optimization](https://mlsanity.com/paper/2102.08363/) (2021), cited by 171 papers here
- [Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble](https://mlsanity.com/paper/2110.01548/) (2021), cited by 150 papers here

## New (17 Sep–7 Oct)

- [Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2610.09763/) (2026-10-07)
- [Role-Adaptive Policy Optimization for Offline Reinforcement Learning](https://mlsanity.com/paper/2609.40149/) (2026-09-30)
- [In-Distribution Imagination for Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2609.38673/) (2026-09-29)
- [Finite-Sample Theory for Fitted Q-Iteration When Actions Are Functions](https://mlsanity.com/paper/2609.36390/) (2026-09-28)
- [Deep Weighted Bellman Residual Minimization for Q^* Estimation](https://mlsanity.com/paper/2609.34593/) (2026-09-28)
- [Policy Plasticity Matters in Offline-to-Online Reinforcement Learning: Refitting Offline Policies for Online Adaptation](https://mlsanity.com/paper/2609.33127/) (2026-09-27)
- [From Static Policies to Adaptive Priors in Offline Reinforcement Learning](https://mlsanity.com/paper/2609.35880/) (2026-09-26)
- [PORL: Pretrained Offline Reinforcement Learning for the Job Shop Scheduling Problem](https://mlsanity.com/paper/2609.30948/) (2026-09-25)
- [Optimal Sequential Annotations for Off-Policy Evaluation](https://mlsanity.com/paper/2609.26707/) (2026-09-22)
- [Lifted Bellman Linear Programming for Offline Reinforcement Learning](https://mlsanity.com/paper/2609.24489/) (2026-09-21)