Towards Fault Tolerance in Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Agent faults pose a significant threat to the performance of multi-agent reinforcement learning (MARL) algorithms, introducing two key challenges. First, agents often struggle to extract critical information from the chaotic state space created by unexpected faults. Second, transitions recorded before and after faults in the replay buffer affect training unevenly, leading to a sample imbalance problem. To overcome these challenges, this paper enhances the fault tolerance of MARL by combining optimized model architecture with a tailored training data sampling strategy. Specifically, an attention mechanism is incorporated into the actor and critic networks to automatically detect faults and dynamically regulate the attention given to faulty agents. Additionally, a prioritization mechanism is introduced to selectively sample transitions critical to current training needs. To further support research in this area, we design and open-source a highly decoupled code platform for fault-tolerant MARL, aimed at improving the efficiency of studying related problems. Experimental results demonstrate the effectiveness of our method in handling various types of faults, faults occurring in any agent, and faults arising at random times.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 67 references that are in the map. First posted on arXiv 30 Nov 2024.
Builds on
35 of 67 references.
References · 35
- 2017Attention Is All You Needcited by 59,932
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2015Prioritized Experience Replaycited by 1,070
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
- 2017Counterfactual Multi-Agent Policy Gradientscited by 733
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
- 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
- 2024Large Language Models: A Surveycited by 506
23 more
- 2014Recurrent Models of Visual Attentioncited by 344
- 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
- 2017Learning from Noisy Labels with Distillationcited by 160
- 2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
- 2017RLlib: Abstractions for Distributed Reinforcement Learningcited by 53
- 2020Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learningcited by 47
- 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
- 2020R-MADDPG for Partially Observable Environments and Limited Communicationcited by 36
- 2023Robust Multi-Agent Reinforcement Learning with State Uncertaintycited by 35
- 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
- 2019Multi-Agent Actor-Critic with Hierarchical Graph Attention Networkcited by 33
- 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 24
- 2020Cooperative Internet of UAVs: Distributed Trajectory Design by Multi-agent Deep Reinforcement Learningcited by 18
- 2023Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Gamecited by 18
- 2021ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxationcited by 17
- 2023Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Explorationcited by 17
- 2024Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priorscited by 14
- 2023Attention-Guided Contrastive Role Representations for Multi-Agent Reinforcement Learningcited by 11
- 2018Multi-agent Deep Reinforcement Learning with Extremely Noisy Observationscited by 10
- 2023Attention-Based Recurrence for Multi-Agent Reinforcement Learning under Stochastic Partial Observabilitycited by 8
- 2024A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocationcited by 6
- 2023Effective Multi-Agent Deep Reinforcement Learning Control with Relative Entropy Regularizationcited by 3
- 2023A GOA-Based Fault-Tolerant Trajectory Tracking Control for an Underwater Vehicle of Multi-Thruster System without Actuator Saturationcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.