mlsanityData to 14 Oct 2026

Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning

arXiv 2112.15578 ↗First posted 31 Dec 2021cs.LG cs.AIcited by 3

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We hypothesize that empirically studying the sample complexity of offline reinforcement learning (RL) is crucial for the practical applications of RL in the real world. Several recent works have demonstrated the ability to learn policies directly from offline data. In this work, we ask the question of the dependency on the number of samples for learning from offline data. Our objective is to emphasize that studying sample complexity for offline RL is important, and is an indicator of the usefulness of existing offline algorithms. We propose an evaluation approach for sample complexity analysis of offline RL.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 14 references that are in the map. First posted on arXiv 31 Dec 2021.

Builds on

10 of 14 references.

Closest ancestors · 7

Same direction, newest first

  1. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  2. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning22 Sep
  3. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  4. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularization14 Mar
  5. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  7. 2019An Optimistic Perspective on Offline Reinforcement Learning10 Jul

Other references · 3

  1. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  2. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  3. 2020Off-Policy Adversarial Inverse Reinforcement Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2023Efficient Deep Reinforcement Learning Requires Regulating Overfittingcited by 36
  2. 2024Improving Offline Reinforcement Learning with Inaccurate Simulatorscited by 4
  3. 2025Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsitycited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2023On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples
  2. 2022Settling the Sample Complexity of Model-Based Offline Reinforcement Learning
  3. 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
  4. 2020What are the Statistical Limits of Offline RL with Linear Function Approximation?
  5. 2022The Role of Coverage in Online Reinforcement Learning
  6. 2025On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures
  7. 2021Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation
  8. 2022Offline Reinforcement Learning with Realizability and Single-policy Concentrability