# Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2501.15495 · HTML: https://mlsanity.com/paper/2501.15495/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Builds on (most cited first, 13 of 13)

- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Deep Reinforcement Learning in Parameterized Action Space](https://mlsanity.com/paper/1511.04143/) (2015)
- [Deep Q-learning from Demonstrations](https://mlsanity.com/paper/1704.03732/) (2017)
- [Pre-training Neural Networks with Human Demonstrations for Deep Reinforcement Learning](https://mlsanity.com/paper/1709.04083/) (2017)
- [Teaching on a Budget in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1905.01357/) (2019)
- [Pre-training with Non-expert Human Demonstration for Deep Reinforcement Learning](https://mlsanity.com/paper/1812.08904/) (2018)
- [A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraint](https://mlsanity.com/paper/2011.14281/) (2020)
- [Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.13085/) (2020)
- [Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systems](https://mlsanity.com/paper/2112.00424/) (2021)
- [Multi-Agent Advisor Q-Learning](https://mlsanity.com/paper/2111.00345/) (2021)

## Abstract (arXiv)

Reinforcement Learning (RL) enables an intelligent agent to optimise its performance in a task by continuously taking action from an observed state and receiving a feedback from the environment in form of rewards. RL typically uses tables or linear approximators to map state-action tuples that maximises the reward. Combining RL with deep neural networks (DRL) significantly increases its scalability and enables it to address more complex problems than before. However, DRL also inherits downsides from both RL and deep learning. Despite DRL improves generalisation across similar state-action pairs when compared to simpler RL policy representations like tabular methods, it still requires the agent to adequately explore the state-action space. Additionally, deep methods require more training data, with the volume of data escalating with the complexity and size of the neural network. As a result, deep RL requires a long time to collect enough agent-environment samples and to successfully learn the underlying policy. Furthermore, often even a slight alteration to the task invalidates any previous acquired knowledge. To address these shortcomings, Transfer Learning (TL) has been introduced, which enables the use of external knowledge from other tasks or agents to enhance a learning process. The goal of TL is to reduce the learning complexity for an agent dealing with an unfamiliar task by simplifying the exploration process. This is achieved by lowering the amount of new information required by its learning model, resulting in a reduced overall convergence time...