mlsanityData to 14 Oct 2026

A Unified Algorithmic Framework for Hybrid Reinforcement Learning in Tabular MDPs with Shifted Transition Dynamics

arXiv 2607.25207 ↗First posted 28 Jul 2026cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper investigates a hybrid reinforcement learning setting in tabular Markov Decision Processes (MDPs), where an agent aims to learn an optimal policy by combining online interactions with a target environment and offline data from a source environment. A central challenge is that offline data may be collected from outdated environments with shifted transition dynamics, making naive integration of historical data ineffective. To address this, we propose a unified algorithmic framework featuring two algorithms: MIN-UCB-VI for regret minimization and MAX-LCB-VI for best policy identification. Both algorithms leverage fine-grained bias information to more effectively exploit offline data under general transition shifts. We provide theoretical guarantees for our framework, including both instance-dependent and independent upper bounds on regret and sub-optimality gap. Furthermore, we establish matching lower bounds to demonstrate the optimality of our approach and validate our theoretical findings through extensive experiments.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 40 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

5 of 40 references.

References · 5

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 161
  4. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 17
  5. 2023Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Provably Safe Sim-to-Real Transfercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Hybrid Reinforcement Learning Breaks Sample Size Barriers in Linear MDPs
  2. 2023Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning
  3. 2026Model-Based Reinforcement Learning with Double Oracle Efficiency in Policy Optimization and Offline Estimation
  4. 2026Minimax Optimal Strategy for Delayed Observations in Online Reinforcement Learning
  5. 2026Offline-Online Reinforcement Learning for Linear Mixture MDPs
  6. 2025Reinforcement Learning for Individual Optimal Policy from Heterogeneous Data
  7. 2026Online Markov Decision Processes with Terminal Law Constraints
  8. 2026Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning