mlsanityData to 14 Oct 2026

Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity

arXiv 2506.17155 ↗First posted 20 Jun 2025cs.LG cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

In this paper, we investigate the use of small datasets in the context of offline reinforcement learning (RL). While many common offline RL benchmarks employ datasets with over a million data points, many offline RL applications rely on considerably smaller datasets. We show that offline RL algorithms can overfit on small datasets, resulting in poor performance. To address this challenge, we introduce "Sparse-Reg": a regularization technique based on sparsity to mitigate overfitting in offline reinforcement learning, enabling effective learning in limited data settings and outperforming state-of-the-art baselines in continuous control.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 35 of its 49 references that are in the map. First posted on arXiv 20 Jun 2025.

Builds on

35 of 49 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023Extreme Q-Learning: MaxEnt RL without Entropy5 Jan
  2. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Library13 Oct
  3. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning31 Jan
  4. 2022The Challenges of Exploration for Offline Reinforcement Learning27 Jan
  5. 2021Single-Shot Pruning for Offline Reinforcement Learning31 Dec
  6. 2021Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning31 Dec
  7. 2021A Dataset Perspective on Offline Reinforcement Learning8 Nov
  8. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  9. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning22 Sep
  10. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  11. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularization14 Mar
  12. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
3 more
  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  2. 2019Behavior Regularized Offline Reinforcement Learning26 Nov
  3. 2019An Optimistic Perspective on Offline Reinforcement Learning10 Jul

Other references · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Layer Normalizationcited by 5,669
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2017Decoupled Weight Decay Regularizationcited by 1,831
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  9. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  10. 2020What is the State of Neural Network Pruning?cited by 583
  11. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  12. 2019The State of Sparsity in Deep Neural Networkscited by 476
8 more
  1. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  2. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 325
  3. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  4. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  5. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  6. 2019DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measurescited by 56
  7. 2016Pruning Convolutional Neural Networks for Resource Efficient Inferencecited by 48
  8. 2019Explicit Group Sparse Projection with Applications to Deep Learning and NMFcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Sparse Masked Attention Policies for Reliable Generalizationcited by 0
  2. 2026Improving Generalization and Robustness in Offline Reinforcement Learning via Boundary-Aware Data Augmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Offline Reinforcement Learning with Corruption Robustness
  2. 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
  3. 2025Fewer May Be Better: Enhancing Offline Reinforcement Learning with Reduced Dataset
  4. 2024Integrating Domain Knowledge for handling Limited Data in Offline RL
  5. 2025Enhancing Robustness of Offline Reinforcement Learning Under Data Corruption via Sharpness-Aware Minimization
  6. 2025On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures
  7. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learning
  8. 2025Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model