mlsanityData to 7 Oct 2026

Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers

arXiv 2006.13916 ↗First posted 24 Jun 2020cs.LG cs.AI cs.ROcited by 93

Not on the map. Why?

Cited by 28 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a simple, practical, and intuitive approach for domain adaptation in reinforcement learning. Our approach stems from the idea that the agent's experience in the source domain should look similar to its experience in the target domain. Building off of a probabilistic view of RL, we formally show that we can achieve this goal by compensating for the difference in dynamics by modifying the reward function. This modified reward function is simple to estimate by learning auxiliary classifiers that distinguish source-domain transitions from target-domain transitions. Intuitively, the modified reward function penalizes the agent for visiting states and taking actions in the source domain which are not possible in the target domain. Said another way, the agent is penalized for transitions that would indicate that the agent is interacting with the source domain, rather than the target domain. Our approach is applicable to domains with continuous states and actions and does not require learning an explicit model of the dynamics. On discrete and continuous control tasks, we illustrate the mechanics of our approach and demonstrate its scalability to high-dimensional tasks.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 10; 2023: 12; 2024: 20; 2025: 24; 2026: 17.

93 papers.

Sort:

By area:

  1. 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 59
  2. 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
  3. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
  4. 2021Mismatched No More: Joint Model-Policy Optimization for Model-Based RLcited by 35
  5. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  6. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  7. 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
  8. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learningcited by 27
  9. 2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
  10. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learningcited by 25
  11. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  12. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
38 more
  1. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  2. 2022Transfer RL across Observation Feature Spaces via Model-Based Regularizationcited by 18
  3. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  4. 2024Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learningcited by 18
  5. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  6. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  7. 2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gapscited by 15
  8. 2024Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learningcited by 15
  9. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 15
  10. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 15
  11. 2024Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learningcited by 13
  12. 2025CTRLS: Chain-of-Thought Reasoning via Latent State-Transitioncited by 13
  13. 2024On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluationcited by 9
  14. 2024xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editingcited by 9
  15. 2024Policy Learning for Off-Dynamics RL with Deficient Supportcited by 8
  16. 2024Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Datacited by 8
  17. 2025Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamicscited by 8
  18. 2025Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Datacited by 8
  19. 2025MOBODY: Model Based Off-Dynamics Offline Reinforcement Learningcited by 8
  20. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 8
  21. 2022Explaining Reinforcement Learning Policies through Counterfactual Trajectoriescited by 7
  22. 2023Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learningcited by 7
  23. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 7
  24. 2025Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPscited by 7
  25. 2025DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learningcited by 7
  26. 2021Fractional Transfer Learning for Deep Model-Based Reinforcement Learningcited by 6
  27. 2021An in-depth experimental study of sensor usage and visual reasoning of robots navigating in real environmentscited by 6
  28. 2023Learning with a Mole: Transferable latent spatial representations for navigation without reconstructioncited by 6
  29. 2021Solving the Real Robot Challenge using Deep Reinforcement Learningcited by 5
  30. 2022Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Taskscited by 5
  31. 2022Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learningcited by 5
  32. 2022Learning Task Embeddings for Teamwork Adaptation in Multi-Agent Reinforcement Learningcited by 5
  33. 2023Marginalized Importance Sampling for Off-Environment Policy Evaluationcited by 5
  34. 2024Learning to navigate efficiently and precisely in real environmentscited by 5
  35. 2024Provable Domain Adaptation for Offline Reinforcement Learning with Limited Samplescited by 5
  36. 2025Active Advantage-Aligned Online Reinforcement Learning with Offline Datacited by 5
  37. 2025Linear Mixture Distributionally Robust Markov Decision Processescited by 5
  38. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 5

50 most cited of 93 All 93 (JSON)