mlsanityData to 14 Oct 2026

Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learning

arXiv 2605.13054 ↗First posted 13 May 2026cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Cross-domain offline reinforcement learning learns a target policy from pre-collected source and target datasets with different dynamics. When target data are scarce, effectively compensating for their limited coverage using source data remains challenging due to the discrepancy between domains. We propose Target-aligned Coverage Expansion (TCE), which leverages source states to generatively realign and expand the limited target support while controlling the generation error induced by this expansion. We further derive a performance gap bound that characterizes the interplay between generation error and source--target dynamics gap, providing theoretical guidance for effective coverage expansion and source utilization. Across diverse cross-domain environments, TCE consistently outperforms state-of-the-art baselines, while further analysis empirically supports the guidance provided by our theoretical findings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 44 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

39 of 44 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026Flow Actor-Critic for Offline Reinforcement Learning20 Feb
  2. 2025Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts2 Dec
  3. 2025DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learning28 Jul
  4. 2025MOBODY: Model Based Off-Dynamics Offline Reinforcement Learning10 Jun
  5. 2024Exclusively Penalized Q-learning for Offline Reinforcement Learning23 May
  6. 2024Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learning10 May
  7. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching4 Feb
  8. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning22 Jun
  9. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learning9 Jun
  10. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learning13 Mar
  11. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  12. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
3 more
  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  3. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  8. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  9. 2019On Variational Bounds of Mutual Informationcited by 559
  10. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  11. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 97
  12. 2018Evaluating Reinforcement Learning Algorithms in Observational Health Settingscited by 60
12 more
  1. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  2. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 41
  3. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 31
  4. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  5. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 25
  6. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  7. 2021Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimizationcited by 18
  8. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 17
  9. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  10. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9
  11. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  12. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filtering
  2. 2026Target-Aligned Bellman Backup for Cross-domain Offline Reinforcement Learning
  3. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correction
  4. 2026Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning
  5. 2026Unifying Value Alignment and Assignment in Cross-Domain Offline Reinforcement Learning with Heterogeneous Datasets
  6. 2026Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning
  7. 2026Bridging Dynamics Gaps via Diffusion Schr"odinger Bridge for Cross-Domain Reinforcement Learning
  8. 2026DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning