# Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2112.15578 · HTML: https://mlsanity.com/paper/2112.15578/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 31 Dec 2021.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)

## Builds on (most cited first, 10 of 10)

- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [Offline Reinforcement Learning with Fisher Divergence Critic Regularization](https://mlsanity.com/paper/2103.08050/) (2021)
- [A Workflow for Offline Model-Free Robotic Reinforcement Learning](https://mlsanity.com/paper/2109.10813/) (2021)
- [An Optimistic Perspective on Offline Reinforcement Learning](https://mlsanity.com/paper/1907.04543/) (2019)
- [Off-Policy Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.01138/) (2020)

## Built on by (most cited first, 3 of 3)

- [Efficient Deep Reinforcement Learning Requires Regulating Overfitting](https://mlsanity.com/paper/2304.10466/) (2023)
- [Improving Offline Reinforcement Learning with Inaccurate Simulators](https://mlsanity.com/paper/2405.04307/) (2024)
- [Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity](https://mlsanity.com/paper/2506.17155/) (2025)

## Abstract (arXiv)

We hypothesize that empirically studying the sample complexity of offline reinforcement learning (RL) is crucial for the practical applications of RL in the real world. Several recent works have demonstrated the ability to learn policies directly from offline data. In this work, we ask the question of the dependency on the number of samples for learning from offline data. Our objective is to emphasize that studying sample complexity for offline RL is important, and is an indicator of the usefulness of existing offline algorithms. We propose an evaluation approach for sample complexity analysis of offline RL.