mlsanityData to 7 Oct 2026

QPLEX: Duplex Dueling Multi-Agent Q-Learning

arXiv 2008.01062 ↗First posted 3 Aug 2020cs.LG cs.AI cs.MAcited by 267

Not on the map. Why?

Abstract (arXiv)

We explore value-based multi-agent reinforcement learning (MARL) in the popular paradigm of centralized training with decentralized execution (CTDE). CTDE has an important concept, Individual-Global-Max (IGM) principle, which requires the consistency between joint and local action selections to support efficient local decision-making. However, in order to achieve scalability, existing MARL methods either limit representation expressiveness of their value function classes or relax the IGM consistency, which may suffer from instability risk or may not perform well in complex domains. This paper presents a novel MARL approach, called duPLEX dueling multi-agent Q-learning (QPLEX), which takes a duplex dueling network architecture to factorize the joint value function. This duplex dueling structure encodes the IGM principle into the neural network architecture and thus enables efficient value function learning. Theoretical analysis shows that QPLEX achieves a complete IGM function class. Empirical experiments on StarCraft II micromanagement tasks demonstrate that QPLEX significantly outperforms state-of-the-art baselines in both online and offline data collection settings, and also reveal that QPLEX achieves high sample efficiency and can benefit from offline datasets without additional online exploration.

Key facts. Cited by 267 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 53; 2023: 52; 2024: 42; 2025: 53; 2026: 38.

267 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  3. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  4. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  5. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  6. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  7. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  8. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  9. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 46
  10. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 42
  11. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
  12. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 36
38 more
  1. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  2. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  3. 2021Offline Decentralized Multi-Agent Reinforcement Learningcited by 32
  4. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  5. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  6. 2021Context-Aware Sparse Deep Coordination Graphscited by 27
  7. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  8. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
  9. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  10. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  11. 2025Embodied AI: From LLMs to World Modelscited by 23
  12. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  13. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  14. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  15. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  16. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  17. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
  18. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  19. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 19
  20. 2021Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learningcited by 18
  21. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 18
  22. 2023Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?cited by 18
  23. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  24. 2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 17
  25. 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
  26. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 16
  27. 2026Intelligent AI Delegationcited by 16
  28. 2021Divergence-Regularized Multi-Agent Actor-Criticcited by 15
  29. 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
  30. 2022ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependencycited by 15
  31. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  32. 2021Self-Organized Polynomial-Time Coordination Graphscited by 14
  33. 2022Multi-Agent Policy Transfer via Task Relationship Modelingcited by 14
  34. 2022QGNN: Value Function Factorisation with Graph Neural Networkscited by 14
  35. 2022Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learningcited by 14
  36. 2021Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignmentcited by 12
  37. 2022Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradientscited by 12
  38. 2022A Survey on Reinforcement Learning Methods in Character Animationcited by 12

50 most cited of 267 All 267 (JSON)