mlsanityData to 14 Oct 2026

Multi-Fidelity Hybrid Reinforcement Learning via Information Gain Maximization

arXiv 2509.14848 ↗First posted 18 Sep 2025cs.LG eess.SPcited by 2

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Optimizing a reinforcement learning (RL) policy typically requires extensive interactions with a high-fidelity simulator of the environment, which are often costly or impractical. Offline RL addresses this problem by allowing training from pre-collected data, but its effectiveness is strongly constrained by the size and quality of the dataset. Hybrid offline-online RL leverages both offline data and interactions with a single simulator of the environment. In many real-world scenarios, however, multiple simulators with varying levels of fidelity and computational cost are available. In this work, we study multi-fidelity hybrid RL for policy optimization under a fixed cost budget. We introduce multi-fidelity hybrid RL via information gain maximization (MF-HRL-IGM), a hybrid offline-online RL algorithm that implements fidelity selection based on information gain maximization through a bootstrapping approach. Theoretical analysis establishes the no-regret property of MF-HRL-IGM, while empirical evaluations demonstrate its superior performance compared to existing benchmarks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 27 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

20 of 27 references.

References · 20

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  4. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  5. 2019Model-Based Reinforcement Learning for Ataricited by 502
  6. 2013A General Framework for Updating Belief Distributionscited by 175
  7. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 97
  8. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 71
  9. 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 60
  10. 2018A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processescited by 52
  11. 2021Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrapcited by 49
  12. 2021DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learningcited by 48
8 more
  1. 2022A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systemscited by 18
  2. 2023A General Offline Reinforcement Learning Framework for Interactive Recommendationcited by 18
  3. 2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gapscited by 17
  4. 2024Bayesian Design Principles for Offline-to-Online Reinforcement Learningcited by 14
  5. 2024Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learningcited by 10
  6. 2022Robust Bayesian Learning for Reliable Wireless AI: Framework and Applicationscited by 9
  7. 2024Improving Offline Reinforcement Learning with Inaccurate Simulatorscited by 4
  8. 2025A Multi-Fidelity Control Variate Approach for Policy Gradient Estimationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Mind the Sim-to-Real Gap & Think Like a Scientistcited by 0
  2. 2026Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Natural Extension To Online Algorithms For Hybrid RL With Limited Coverage
  2. 2025Augmenting Online RL with Offline Data is All You Need: A Unified Hybrid RL Algorithm Design and Analysis
  3. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
  4. 2020MOReL : Model-Based Offline Reinforcement Learning
  5. 2025EXPO: Stable Reinforcement Learning with Expressive Policies
  6. 2025Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL
  7. 2024Hybrid Reinforcement Learning Breaks Sample Size Barriers in Linear MDPs
  8. 2023Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learning