mlsanityData to 7 Oct 2026

Sample Efficient Active Algorithms for Offline Reinforcement Learning

arXiv 2602.01260 ↗First posted 1 Feb 2026cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline reinforcement learning (RL) enables policy learning from static data but often suffers from poor coverage of the state-action space and distributional shift problems. This problem can be addressed by allowing limited online interactions to selectively refine uncertain regions of the learned value function, which is referred to as Active Reinforcement Learning (ActiveRL). While there has been good empirical success, no theoretical analysis is available in the literature. We fill this gap by developing a rigorous sample-complexity analysis of ActiveRL through the lens of Gaussian Process (GP) uncertainty modeling. In this respect, we propose an algorithm and using GP concentration inequalities and information-gain bounds, we derive high-probability guarantees showing that an ε-optimal policy can be learned with O(1/ε2) active transitions, improving upon the Omega(1/ε2(1-γ)4) rate of purely offline methods. Our results reveal that ActiveRL achieves near-optimal information efficiency, that is, guided uncertainty reduction leads to accelerated value-function convergence with minimal online data. Our analysis builds on GP concentration inequalities and information-gain bounds, bridging Bayesian nonparametric regression and reinforcement learning theories. We conduct several experiments to validate the algorithm and theoretical findings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 16 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

12 of 16 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage13 Jul
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May

Other references · 10

  1. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  2. 2017On Kernelized Multi-armed Banditscited by 348
  3. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  4. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  5. 2023Sample Complexity of Kernel-Based Q-Learningcited by 8
  6. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulatorcited by 7
  7. 2023Optimal Sample Complexity of Stable Discounted Markov Decision Processescited by 6
  8. 2021Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithmcited by 2
  9. 2024Active Reinforcement Learning Strategies for Offline Policy Improvementcited by 2
  10. 2023Reinforcement Learning under External Influence: Guarantees, Algorithms, and Sample Complexitycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
  2. 2023Achieving the Asymptotically Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approach
  3. 2022Settling the Sample Complexity of Model-Based Offline Reinforcement Learning
  4. 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
  5. 2020Expert-Supervised Reinforcement Learning for Offline Policy Learning and Evaluation
  6. 2020What are the Statistical Limits of Offline RL with Linear Function Approximation?
  7. 2023On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples
  8. 2023Offline Reinforcement Learning with Additional Covering Distributions