mlsanityData to 14 Oct 2026

DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning

arXiv 2607.16090 ↗First posted 17 Jul 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Transferring policies across domains poses a vital challenge in reinforcement learning, due to the dynamics mismatch between the source and target domains. In this paper, we consider the setting of online dynamics adaptation, where policies are trained in the source domain with sufficient data, while only limited interactions with the target domain are allowed. There are a few existing works that address the dynamics mismatch by employing domain classifiers, value-guided data filtering, or representation learning. Instead, we study the domain adaptation problem from a generative modeling perspective. Specifically, we introduce DADiff, a diffusion-based framework that leverages the discrepancy between source and target domain generative trajectories in the generation process of the next state to estimate the dynamics mismatch. Both reward modification and data selection variants are developed to adapt the policy to the target domain. We also provide a theoretical analysis to show that the performance difference of a given policy between the two domains is bounded by the generative trajectory deviation. More discussions on the applicability of the variants and the connection between our theoretical analysis and the prior work are further provided. We conduct extensive experiments in environments with various shifts to validate the effectiveness of our method. The results demonstrate that our method provides superior performance compared to existing approaches, effectively addressing the dynamics mismatch. We provide the code of our method at https://github.com/hanyang-chen/DADiff-release

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 35 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

32 of 35 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learning28 Oct
  2. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatch24 May
  3. 2024Policy Learning for Off-Dynamics RL with Deficient Support16 Feb

Other references · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2016OpenAI Gymcited by 1,736
  5. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  8. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  9. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  10. 2019Active Domain Randomizationcited by 170
  11. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 157
  12. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 140
17 more
  1. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 115
  2. 2023Synthetic Experience Replaycited by 101
  3. 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 97
  4. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 94
  5. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 75
  6. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  7. 2021Cross-domain Imitation from Observationscited by 42
  8. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 38
  9. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 31
  10. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  11. 2024Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learningcited by 18
  12. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 17
  13. 2019Robust Visual Domain Randomization for Reinforcement Learningcited by 13
  14. 2022Policy Adaptation from Foundation Model Feedbackcited by 10
  15. 2025DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learningcited by 7
  16. 2022Zero-Shot Policy Transfer with Disentangled Task Representation of Meta-Reinforcement Learningcited by 5
  17. 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correction
  2. 2026DADP: Domain Adaptive Diffusion Policy
  3. 2026Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning
  4. 2026Bridging Dynamics Gaps via Diffusion Schr"odinger Bridge for Cross-Domain Reinforcement Learning
  5. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing
  6. 2025In-Context Policy Adaptation via Cross-Domain Skill Diffusion
  7. 2024Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective
  8. 2026Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning