mlsanityData to 7 Oct 2026

Robust Transfer Learning with Side Information

arXiv 2603.07921 ↗First posted 9 Mar 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Robust Markov Decision Processes (MDPs) address environmental shift through distributionally robust optimization (DRO) by finding an optimal worst-case policy within an uncertainty set of transition kernels. However, standard DRO approaches require enlarging the uncertainty set under large shifts, which leads to overly conservative and pessimistic policies. In this paper, we propose a framework for transfer under environment shift that derives a robust target-domain policy via estimate-centered uncertainty sets, constructed through constrained estimation that integrates limited target samples with side information about the source-target dynamics. The side information includes bounds on feature moments, distributional distances, and density ratios, yielding improved kernel estimates and tighter uncertainty sets. The side information includes bounds on feature moments, distributional distances, and density ratios, yielding improved kernel estimates and tighter uncertainty sets. Error bounds and convergence results are established for both robust and non-robust value functions. Moreover, we provide a finite-sample guarantee on the learned robust policy and analyze the robust sub-optimality gap. Under mild low-dimensional structure on the transition model, the side information reduces this gap and improves sample efficiency. We assess the performance of our approach across OpenAI Gym environments and classic control problems, consistently demonstrating superior target-domain performance over state-of-the-art robust and non-robust baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 50 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

25 of 50 references.

References · 25

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2016OpenAI Gymcited by 1,684
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  6. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  7. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  8. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 176
  9. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 160
  10. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
  11. 2017Reinforcement Learning under Model Mismatchcited by 52
  12. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
13 more
  1. 2020Partial Policy Iteration for L1-Robust Markov Decision Processescited by 49
  2. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
  3. 2015Distributionally Robust Counterpart in Markov Decision Processescited by 32
  4. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  5. 2023Sample Complexity of Variance-reduced Distributionally Robust Q-learningcited by 28
  6. 2021Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptoticscited by 27
  7. 2023Robust Average-Reward Markov Decision Processescited by 22
  8. 2023Model-Free Robust Average-Reward Reinforcement Learningcited by 20
  9. 2018Importance Weighted Transfer of Samples in Reinforcement Learningcited by 16
  10. 2020Minimax Optimal Estimation of KL Divergence for Continuous Distributionscited by 11
  11. 2024Model-Free Robust Reinforcement Learning with Sample Complexity Analysiscited by 9
  12. 2022Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictionscited by 8
  13. 2023Achieving the Asymptotically Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approachcited by 3

Built on by

Not cited yet.