# Towards Resilience for Multi-Agent QD-Learning

arXiv: https://arxiv.org/abs/2104.03153 · HTML: https://mlsanity.com/paper/2104.03153/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2021.

## Built on by (most cited first, 4 of 4)

- [Resilient Consensus-based Multi-agent Reinforcement Learning with Function Approximation](https://mlsanity.com/paper/2111.06776/) (2021)
- [Risk-Aware Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.02005/) (2023)
- [BARD-MARL: Byzantine-Agent Detection for Learned Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2606.20701/) (2026)
- [Fully Byzantine-Resilient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2609.25701/) (2026)

## Abstract (arXiv)

This paper considers the multi-agent reinforcement learning (MARL) problem for a networked (peer-to-peer) system in the presence of Byzantine agents. We build on an existing distributed Q-learning algorithm, and allow certain agents in the network to behave in an arbitrary and adversarial manner (as captured by the Byzantine attack model). Under the proposed algorithm, if the network topology is (2F+1)-robust and up to F Byzantine agents exist in the neighborhood of each regular agent, we establish the almost sure convergence of all regular agents' value functions to the neighborhood of the optimal value function of all regular agents. For each state, if the optimal Q-values of all regular agents corresponding to different actions are sufficiently separated, our approach allows each regular agent to learn the optimal policy for all regular agents.