mlsanityData to 14 Oct 2026

Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review

arXiv 2411.10268 ↗First posted 15 Nov 2024cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL) is a sub-domain of machine learning, mainly concerned with solving sequential decision-making problems by a learning agent that interacts with the decision environment to improve its behavior through the reward it receives from the environment. This learning paradigm is, however, well-known for being time-consuming due to the necessity of collecting a large amount of data, making RL suffer from sample inefficiency and difficult generalization. Furthermore, the construction of an explicit reward function that accounts for the trade-off between multiple desiderata of a decision problem is often a laborious task. These challenges have been recently addressed utilizing transfer and inverse reinforcement learning (T-IRL). In this regard, this paper is devoted to a comprehensive review of realizing the sample efficiency and generalization of RL algorithms through T-IRL. Following a brief introduction to RL, the fundamental T-IRL methods are presented and the most recent advancements in each research field have been extensively reviewed. Our findings denote that a majority of recent research works have dealt with the aforementioned challenges by utilizing human-in-the-loop and sim-to-real strategies for the efficient transfer of knowledge from source domains to the target domain under the transfer learning scheme. Under the IRL structure, training schemes that require a low number of experience transitions and extension of such frameworks to multi-agent and multi-intention problems have been the priority of researchers in recent years.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 107 of its 218 references that are in the map. First posted on arXiv 15 Nov 2024.

Builds on

107 of 218 references.

Closest ancestors · 14

Same direction, newest first

  1. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching9 Sep
  2. 2021Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts2 Sep
  3. 2021Deep Adaptive Multi-Intention Inverse Reinforcement Learning14 Jul
  4. 2021LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning3 Jun
  5. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks23 Mar
  6. 2021A Lower Bound for the Sample Complexity of Inverse Reinforcement Learning7 Mar
  7. 2021Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees16 Feb
  8. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matching9 Nov
  9. 2020Bayesian Inverse Reinforcement Learning for Collective Animal Movement8 Sep
  10. 2020Off-Policy Adversarial Inverse Reinforcement Learning3 May
  11. 2020Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic24 Feb
  12. 2019Multi-Agent Adversarial Inverse Reinforcement Learning30 Jul
2 more
  1. 2019On the Correctness and Sample Complexity of Inverse Reinforcement Learning2 Jun
  2. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 93

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2022Generative Adversarial Networkscited by 5,627
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2015Continuous control with deep reinforcement learningcited by 3,407
  10. 2015Trust Region Policy Optimizationcited by 2,580
  11. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  12. 2016Generative Adversarial Imitation Learningcited by 1,597
38 more
  1. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  3. 2015Prioritized Experience Replaycited by 1,099
  4. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  5. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  6. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  8. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  10. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  11. 2018Distributed Prioritized Experience Replaycited by 269
  12. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  13. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  14. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 190
  15. 2016Sample Efficient Actor-Critic with Experience Replaycited by 188
  16. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  17. 2014Statistical guarantees for the EM algorithm: From population to sample-based analysiscited by 133
  18. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  19. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 106
  20. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  21. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  22. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  23. 2020Data-efficient Domain Randomization with Bayesian Optimizationcited by 47
  24. 2018Observe and Look Further: Achieving Consistent Performance on Ataricited by 44
  25. 2019Federated Transfer Reinforcement Learning for Autonomous Drivingcited by 44
  26. 2020Experience Replay with Likelihood-free Importance Weightscited by 42
  27. 2020Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentationcited by 42
  28. 2018DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedbackcited by 41
  29. 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
  30. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 39
  31. 2018Sample-Efficient Imitation Learning via Generative Adversarial Netscited by 36
  32. 2019Certified Reinforcement Learning with Logic Guidancecited by 36
  33. 2018VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Controlcited by 35
  34. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Drivingcited by 34
  35. 2022Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximationcited by 31
  36. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  37. 2019Curiosity-Driven Experience Prioritization via Density Estimationcited by 27
  38. 2021SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learningcited by 27

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMscited by 16
  2. 2025Bundle Network: a Machine Learning-Based Bundle Methodcited by 0
  3. 2026Mitigating Data Scarcity in Spaceflight Applications for Offline Reinforcement Learning Using Physics-Informed Deep Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Statistical and Algorithmic Foundations of Reinforcement Learning
  2. 2025Reinforcement Learning from Human Feedback
  3. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  4. 2023Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
  5. 2024Introduction to Reinforcement Learning
  6. 2024A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges
  7. 2022On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency
  8. 2021A Generalised Inverse Reinforcement Learning Framework