# Teaching on a Budget in Multi-Agent Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1905.01357 · HTML: https://mlsanity.com/paper/1905.01357/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2019.

## Built on by (most cited first, 14 of 14)

- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Action Advising with Advice Imitation in Deep Reinforcement Learning](https://mlsanity.com/paper/2104.08441/) (2021)
- [Explainable Action Advising for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2211.07882/) (2022)
- [Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systems](https://mlsanity.com/paper/2112.00424/) (2021)
- [Introspective Action Advising for Interpretable Transfer Learning](https://mlsanity.com/paper/2306.12314/) (2023)
- [Scalable Interactive Machine Learning for Future Command and Control](https://mlsanity.com/paper/2402.06501/) (2024)
- [Learning on a Budget via Teacher Imitation](https://mlsanity.com/paper/2104.08440/) (2021)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.01170/) (2023)
- [Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learning](https://mlsanity.com/paper/2311.16807/) (2023)
- [Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios](https://mlsanity.com/paper/2405.14199/) (2024)
- [Personalized Decision Supports based on Theory of Mind Modeling and Explainable Reinforcement Learning](https://mlsanity.com/paper/2312.08397/) (2023)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.15495/) (2025)
- [A Computational Model of Inclusive Pedagogy: From Understanding to Application](https://mlsanity.com/paper/2505.02853/) (2025)

## Abstract (arXiv)

Deep Reinforcement Learning (RL) algorithms can solve complex sequential decision tasks successfully. However, they have a major drawback of having poor sample efficiency which can often be tackled by knowledge reuse. In Multi-Agent Reinforcement Learning (MARL) this drawback becomes worse, but at the same time, a new set of opportunities to leverage knowledge are also presented through agent interactions. One promising approach among these is peer-to-peer action advising through a teacher-student framework. Despite being introduced for single-agent RL originally, recent studies show that it can also be applied to multi-agent scenarios with promising empirical results. However, studies in this line of research are currently very limited. In this paper, we propose heuristics-based action advising techniques in cooperative decentralised MARL, using a nonlinear function approximation based task-level policy. By adopting Random Network Distillation technique, we devise a measurement for agents to assess their knowledge in any given state and be able to initiate the teacher-student dynamics with no prior role assumptions. Experimental results in a gridworld environment show that such an approach may indeed be useful and needs to be further investigated.