Data to 7 Oct 2026

Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning

arXiv 1906.04737 ↗First posted 11 Jun 2019cs.LG cs.AI cs.MAcited by 90

Not on the map. Why?

Abstract (arXiv)

Recent developments in deep reinforcement learning are concerned with creating decision-making agents which can perform well in various complex domains. A particular approach which has received increasing attention is multi-agent reinforcement learning, in which multiple agents learn concurrently to coordinate their actions. In such multi-agent environments, additional learning problems arise due to the continually changing decision-making policies of agents. This paper surveys recent works that address the non-stationarity problem in multi-agent deep reinforcement learning. The surveyed methods range from modifications in the training procedure, such as centralized training, to learning representations of the opponent's policy, meta-learning, communication, and decentralized learning. The survey concludes with a list of open problems and possible lines of future research.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 16; 2023: 19; 2024: 12; 2025: 11; 2026: 15.

90 papers.

Sort:

By area:

2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
2022A Survey on Model-based Reinforcement Learningcited by 65
2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
2022A Survey of Ad Hoc Teamwork Researchcited by 41
2022Multi-agent Dynamic Algorithm Configurationcited by 36
2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
2022Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workerscited by 28
2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
2024K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoningcited by 21
2021Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learningcited by 20

38 more

2022Influencing Long-Term Behavior in Multiagent Reinforcement Learningcited by 18
2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
2025Virtual Agent Economiescited by 17
2021Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Trainingcited by 16
2021Coordinated Proximal Policy Optimizationcited by 16
2021Causal Multi-Agent Reinforcement Learning: Review and Open Problemscited by 16
2021RLOps: Development Life-cycle of Reinforcement Learning Aided Open RANcited by 15
2022ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependencycited by 15
2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
2022Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learningcited by 14
2021Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systemscited by 12
2022Trust Region Bounds for Decentralized PPO Under Non-stationaritycited by 11
2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 11
2021Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policiescited by 10
2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
2024Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluationcited by 9
2023Improving Intrinsic Exploration by Creating Stationary Objectivescited by 7
2024Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learningcited by 7
2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
2021Dealing with Non-Stationarity in MARL via Trust-Region Decompositioncited by 5
2021Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environmentscited by 5
2022Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibriacited by 5
2023Ensemble Value Functions for Efficient Exploration in Multi-Agent Reinforcement Learningcited by 5
2021Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspectivecited by 4
2021The Holy Grail of Multi-Robot Planning: Learning to Generate Online-Scalable Solutions from Offline-Optimal Expertscited by 4
2023MABL: Bi-Level Latent-Variable World Model for Sample-Efficient Multi-Agent Reinforcement Learningcited by 4
2024Crafting Desirable Climate Trajectories with RL Explored Socio-Environmental Simulationscited by 4
2021Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarioscited by 3
2021Sequential memory improves sample and memory efficiency in Episodic Controlcited by 3
2022Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behaviorcited by 3
2022Deep Reinforcement Learning for Multi-Agent Interactioncited by 3
2023Learning Complex Teamwork Tasks Using a Given Sub-task Decompositioncited by 3
2023On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learningcited by 3
2023Robust Multi-agent Communication via Multi-view Message Certificationcited by 3
2023Decentralized Multi-Agent Reinforcement Learning with Global State Predictioncited by 3
2024Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machinescited by 3
2025A Principle of Targeted Intervention for Multi-Agent Reinforcement Learningcited by 3

50 most cited of 90 All 90 (JSON)