# A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraint

arXiv: https://arxiv.org/abs/2011.14281 · HTML: https://mlsanity.com/paper/2011.14281/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2020.

## Built on by (most cited first, 7 of 7)

- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Hiding in Plain Sight: Differential Privacy Noise Exploitation for Evasion-resilient Localized Poisoning Attacks in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2307.00268/) (2023)
- [A Multi-Agent Multi-Environment Mixed Q-Learning for Partially Decentralized Wireless Network Optimization](https://mlsanity.com/paper/2409.16450/) (2024)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.01170/) (2023)
- [BRNES: Enabling Security and Privacy-aware Experience Sharing in Multiagent Robotic and Autonomous Systems](https://mlsanity.com/paper/2308.01274/) (2023)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.15495/) (2025)
- [CCKS: Consensus-based Communication and Knowledge Sharing](https://mlsanity.com/paper/2606.12281/) (2026)

## Abstract (arXiv)

In teacher-student framework, a more experienced agent (teacher) helps accelerate the learning of another agent (student) by suggesting actions to take in certain states. In cooperative multiagent reinforcement learning (MARL), where agents need to cooperate with one another, a student may fail to cooperate well with others even by following the teachers' suggested actions, as the polices of all agents are ever changing before convergence. When the number of times that agents communicate with one another is limited (i.e., there is budget constraint), the advising strategy that uses actions as advices may not be good enough. We propose a partaker-sharer advising framework (PSAF) for cooperative MARL agents learning with budget constraint. In PSAF, each Q-learner can decide when to ask for Q-values and share its Q-values. We perform experiments in three typical multiagent learning problems. Evaluation results show that our approach PSAF outperforms existing advising methods under both unlimited and limited budget, and we give an analysis of the impact of advising actions and sharing Q-values on agents' learning.