# Adaptive Policy Transfer in Reinforcement Learning

arXiv: https://arxiv.org/abs/2105.04699 · HTML: https://mlsanity.com/paper/2105.04699/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost](https://mlsanity.com/paper/1810.06045/) (2018)
- [Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model](https://mlsanity.com/paper/1610.03518/) (2016)
- [Learning Contact-Rich Manipulation Skills with Guided Policy Search](https://mlsanity.com/paper/1501.05611/) (2015)
- [Learning Transferable Policies for Monocular Reactive MAV Control](https://mlsanity.com/paper/1608.00627/) (2016)
- [Mutual Alignment Transfer Learning](https://mlsanity.com/paper/1707.07907/) (2017)
- [Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control](https://mlsanity.com/paper/1712.03303/) (2017)
- [Cross-Domain Transfer in Reinforcement Learning using Target Apprentice](https://mlsanity.com/paper/1801.06920/) (2018)

## Built on by (most cited first, 2 of 2)

- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Near-Field Spot Beamfocusing: A Correlation-Aware Transfer Learning Approach](https://mlsanity.com/paper/2405.19347/) (2024)

## Abstract (arXiv)

Efficient and robust policy transfer remains a key challenge for reinforcement learning to become viable for real-wold robotics. Policy transfer through warm initialization, imitation, or interacting over a large set of agents with randomized instances, have been commonly applied to solve a variety of Reinforcement Learning tasks. However, this seems far from how skill transfer happens in the biological world: Humans and animals are able to quickly adapt the learned behaviors between similar tasks and learn new skills when presented with new situations. Here we seek to answer the question: Will learning to combine adaptation and exploration lead to a more efficient transfer of policies between domains? We introduce a principled mechanism that can "Adapt-to-Learn", that is adapt the source policy to learn to solve a target task with significant transition differences and uncertainties. We show that the presented method learns to seamlessly combine learning from adaptation and exploration and leads to a robust policy transfer algorithm with significantly reduced sample complexity in transferring skills between related tasks.