# An Optimal Online Method of Selecting Source Policies for Reinforcement Learning

arXiv: https://arxiv.org/abs/1709.08201 · HTML: https://mlsanity.com/paper/1709.08201/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2017.

## Built on by (most cited first, 13 of 13)

- [Model-Based Transfer Learning for Contextual Reinforcement Learning](https://mlsanity.com/paper/2408.04498/) (2024)
- [A Taxonomy of Similarity Metrics for Markov Decision Processes](https://mlsanity.com/paper/2103.04706/) (2021)
- [CUP: Critic-Guided Policy Reuse](https://mlsanity.com/paper/2210.08153/) (2022)
- [Discovering Generalizable Skills via Automated Generation of Diverse Tasks](https://mlsanity.com/paper/2106.13935/) (2021)
- [Lifetime policy reuse and the importance of task capacity](https://mlsanity.com/paper/2106.01741/) (2021)
- [Efficient Bayesian Policy Reuse with a Scalable Observation Model in Deep Reinforcement Learning](https://mlsanity.com/paper/2204.07729/) (2022)
- [Multi-trainer Interactive Reinforcement Learning System](https://mlsanity.com/paper/2210.08050/) (2022)
- [Multi-agent Policy Reciprocity with Theoretical Guarantee](https://mlsanity.com/paper/2304.05632/) (2023)
- [On the Value of Myopic Behavior in Policy Reuse](https://mlsanity.com/paper/2305.17623/) (2023)
- [IOB: Integrating Optimization Transfer and Behavior Transfer for Multi-Policy Reuse](https://mlsanity.com/paper/2308.07351/) (2023)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)
- [Contextual Policy Transfer in Reinforcement Learning Domains via Deep Mixtures-of-Experts](https://mlsanity.com/paper/2003.00203/) (2020)
- [Enhancing Traffic Signal Control through Model-based Reinforcement Learning and Policy Reuse](https://mlsanity.com/paper/2503.08728/) (2025)

## Abstract (arXiv)

Transfer learning significantly accelerates the reinforcement learning process by exploiting relevant knowledge from previous experiences. The problem of optimally selecting source policies during the learning process is of great importance yet challenging. There has been little theoretical analysis of this problem. In this paper, we develop an optimal online method to select source policies for reinforcement learning. This method formulates online source policy selection as a multi-armed bandit problem and augments Q-learning with policy reuse. We provide theoretical guarantees of the optimal selection process and convergence to the optimal policy. In addition, we conduct experiments on a grid-based robot navigation domain to demonstrate its efficiency and robustness by comparing to the state-of-the-art transfer learning method.