# Context-Aware Policy Reuse

arXiv: https://arxiv.org/abs/1806.03793 · HTML: https://mlsanity.com/paper/1806.03793/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2018.

## Built on by (most cited first, 16 of 16)

- [Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress](https://mlsanity.com/paper/2206.01626/) (2022)
- [Reinforcement Learning in Presence of Discrete Markovian Context Evolution](https://mlsanity.com/paper/2202.06557/) (2022)
- [CUP: Critic-Guided Policy Reuse](https://mlsanity.com/paper/2210.08153/) (2022)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [Discovering Generalizable Skills via Automated Generation of Diverse Tasks](https://mlsanity.com/paper/2106.13935/) (2021)
- [Lifetime policy reuse and the importance of task capacity](https://mlsanity.com/paper/2106.01741/) (2021)
- [Efficient Bayesian Policy Reuse with a Scalable Observation Model in Deep Reinforcement Learning](https://mlsanity.com/paper/2204.07729/) (2022)
- [Multi-agent Policy Reciprocity with Theoretical Guarantee](https://mlsanity.com/paper/2304.05632/) (2023)
- [On the Value of Myopic Behavior in Policy Reuse](https://mlsanity.com/paper/2305.17623/) (2023)
- [IOB: Integrating Optimization Transfer and Behavior Transfer for Multi-Policy Reuse](https://mlsanity.com/paper/2308.07351/) (2023)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)
- [CARoL: Context-aware Adaptation for Robot Learning](https://mlsanity.com/paper/2506.07006/) (2025)
- [Multi-Agent Reinforcement Learning and Real-Time Decision-Making in Robotic Soccer for Virtual Environments](https://mlsanity.com/paper/2512.03166/) (2025)
- [Contextual Policy Transfer in Reinforcement Learning Domains via Deep Mixtures-of-Experts](https://mlsanity.com/paper/2003.00203/) (2020)
- [Combining Trained Models in Reinforcement Learning](https://mlsanity.com/paper/2605.02159/) (2026)
- [How to Find and Reuse Policies for Continuous Adaptation in Lifelong Reinforcement Learning](https://mlsanity.com/paper/2610.03119/) (2026)

## Abstract (arXiv)

Transfer learning can greatly speed up reinforcement learning for a new task by leveraging policies of relevant tasks. Existing works of policy reuse either focus on only selecting a single best source policy for transfer without considering contexts, or cannot guarantee to learn an optimal policy for a target task. To improve transfer efficiency and guarantee optimality, we develop a novel policy reuse method, called Context-Aware Policy reuSe (CAPS), that enables multi-policy transfer. Our method learns when and which source policy is best for reuse, as well as when to terminate its reuse. CAPS provides theoretical guarantees in convergence and optimality for both source policy selection and target task learning. Empirical results on a grid-based navigation domain and the Pygame Learning Environment demonstrate that CAPS significantly outperforms other state-of-the-art policy reuse methods.