mlsanityData to 14 Oct 2026

Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review

arXiv 2404.17687 ↗First posted 26 Apr 2024cs.LG cs.AI cs.ROcited by 2

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement Learning (RL) provides a framework in which agents can be trained, via trial and error, to solve complex decision-making problems. Learning with little supervision causes RL methods to require large amounts of data, rendering them too expensive for many applications (e.g., robotics). By reusing knowledge from a different task, knowledge transfer methods present an alternative to reduce the training time in RL. Given the severe data scarcity, due to their flexibility, there has been a growing interest in methods capable of transferring knowledge across different domains (i.e., problems with different representations). However, identifying similarities and adapting knowledge across tasks from different domains requires matching their representations or finding domain-invariant features. These processes can be data-demanding, which poses the main challenge in cross-domain knowledge transfer: to select and transform knowledge in a data-efficient way, such that it accelerates learning in the target task, despite the presence of significant differences across problems (e.g., robots with distinct morphologies). Thus, this review presents a unifying analysis of methods focused on transferring knowledge across different domains. Through a taxonomy based on a transfer-approach categorization and a characterization of works based on their data-assumption requirements, the contributions of this article are 1) a comprehensive and systematic revision of knowledge transfer methods for the cross-domain RL setting, 2) a categorization and characterization of such methods to provide an analysis based on relevant features such as their transfer approach and data requirements, and 3) a discussion on the main challenges regarding cross-domain knowledge transfer, as well as on ideas of future directions worth exploring to address these problems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 65 of its 179 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

65 of 179 references.

References · 65

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016OpenAI Gymcited by 1,736
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  9. 2018DeepMind Control Suitecited by 956
  10. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  11. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  12. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
38 more
  1. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  3. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  4. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  5. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  6. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  7. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  8. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  9. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  10. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  11. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  12. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  13. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 216
  14. 2019Temporal Cycle-Consistency Learningcited by 154
  15. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  16. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  17. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 127
  18. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  19. 2016Resnet in Resnet: Generalizing Residual Architecturescited by 110
  20. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 107
  21. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 87
  22. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 85
  23. 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 81
  24. 2020Quantifying Differences in Reward Functionscited by 63
  25. 2020Learning Robot Skills with Temporal Variational Inferencecited by 54
  26. 2020MushroomRL: Simplifying Reinforcement Learning Researchcited by 50
  27. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  28. 2018Horizon: Facebook's Open Source Applied Reinforcement Learning Platformcited by 48
  29. 2020Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interferencecited by 47
  30. 2020Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistencycited by 43
  31. 2021Cross-domain Imitation from Observationscited by 42
  32. 2019P3O: Policy-on Policy-off Policy Optimizationcited by 37
  33. 2024Mirage: Cross-Embodiment Zero-Shot Policy Transfer with Cross-Paintingcited by 36
  34. 2020The MAGICAL Benchmark for Robust Imitationcited by 32
  35. 2020Hierarchically Decoupled Imitation for Morphological Transfercited by 31
  36. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  37. 2017Black-Box Data-efficient Policy Search for Roboticscited by 14
  38. 2020REPAINT: Knowledge Transfer in Deep Reinforcement Learningcited by 14

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Transferable Deep Reinforcement Learning for Cross-Domain Navigation: from Farmland to the Mooncited by 0
  2. 2026Cognition-Inspired Dual-Stream Semantic Enhancement for Vision-Based Dynamic Emotion Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Transfer RL via the Undo Maps Formalism
  2. 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities
  3. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
  4. 2023Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges
  5. 2024Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloning
  6. 2024Integrating Domain Knowledge for handling Limited Data in Offline RL
  7. 2025In-Context Policy Adaptation via Cross-Domain Skill Diffusion
  8. 2024Cross Domain Policy Transfer with Effect Cycle-Consistency