mlsanityData to 7 Oct 2026

Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics

arXiv 1910.01465 ↗First posted 3 Oct 2019cs.LG cs.AI cs.MAcited by 46

Not on the map. Why?

Abstract (arXiv)

Many real world tasks require multiple agents to work together. Multi-agent reinforcement learning (RL) methods have been proposed in recent years to solve these tasks, but current methods often fail to efficiently learn policies. We thus investigate the presence of a common weakness in single-agent RL, namely value function overestimation bias, in the multi-agent setting. Based on our findings, we propose an approach that reduces this bias by using double centralized critics. We evaluate it on six mixed cooperative-competitive tasks, showing a significant advantage over current methods. Finally, we investigate the application of multi-agent methods to high-dimensional robotic tasks and show that our approach can be used to learn decentralized policies in this domain.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 4; 2023: 8; 2024: 10; 2025: 9; 2026: 10.

46 papers.

Sort:

By area:

  1. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  2. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  3. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  4. 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 52
  5. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 36
  6. 2022Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregationcited by 34
  7. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 23
  8. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  9. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
  10. 2024Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectoriescited by 9
  11. 2025LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systemscited by 8
  12. 2023Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learningcited by 7
34 more
  1. 2025Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learningcited by 7
  2. 2021MDPGT: Momentum-based Decentralized Policy Gradient Trackingcited by 6
  3. 2022Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Reviewcited by 6
  4. 2025LEED: A Highly Efficient and Scalable LLM-Empowered Expert Demonstrations Framework for Multi-Agent Reinforcement Learningcited by 6
  5. 2022Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformercited by 5
  6. 2024Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learningcited by 5
  7. 2023Self-Confirming Transformer for Belief-Conditioned Adaptation in Offline Multi-Agent Reinforcement Learningcited by 4
  8. 2024Putting Data at the Centre of Offline Multi-Agent Reinforcement Learningcited by 4
  9. 2023Decentralized Multi-Agent Reinforcement Learning with Global State Predictioncited by 3
  10. 2023Effective Multi-Agent Deep Reinforcement Learning Control with Relative Entropy Regularizationcited by 3
  11. 2024Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Controlcited by 2
  12. 2024Temporal Prototype-Aware Learning for Active Voltage Control on Power Distribution Networkscited by 2
  13. 2026Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policiescited by 2
  14. 2026TRIDENT: Breaking the Hybrid-Safety-Physics Coupling for Provably Safe Multi-Agent Reinforcement Learningcited by 2
  15. 2021Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competitioncited by 1
  16. 2022Transferable Multi-Agent Reinforcement Learning with Dynamic Participating Agentscited by 1
  17. 2024Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Trainingcited by 1
  18. 2025B3C: A Minimalist Approach to Offline Multi-Agent Reinforcement Learningcited by 1
  19. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demoscited by 1
  20. 2026Prism: Spectral Parameter Sharing for Multi-Agent Reinforcement Learningcited by 1
  21. 2023Promoting Cooperation in Multi-Agent Reinforcement Learning via Mutual Helpcited by 0
  22. 2024Robustifying a Policy in Multi-Agent RL with Diverse Cooperative Behaviors and Adversarial Style Sampling for Assistive Taskscited by 0
  23. 2024Feint Behaviors and Strategies: Formalization, Implementation and Evaluationcited by 0
  24. 2025Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent Systemcited by 0
  25. 2025Emergent Heterogeneous Swarm Control Through Hebbian Learningcited by 0
  26. 2025SafeDiver: Cooperative AUV-USV Assisted Diver Communication via Multi-agent Reinforcement Learning Approachcited by 0
  27. 2025Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspectivecited by 0
  28. 2026PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillationcited by 0
  29. 2026Individual Control Barrier Functions-Guided Diffusion Model for Safe Offline Multi-Agent Reinforcement Learningcited by 0
  30. 2026Multi-Agent Reinforcement Learning via Agent-Specific Preferencecited by 0
  31. 2026Multi-AUV Ad-hoc network-based Target Tracking: A Value Gradient Guidance Multi-Agent Diffusion Reinforcement Learning Approachcited by 0
  32. 2026Low-Altitude Fluid Antenna Network with Multi-Agent Reinforcement Learningcited by 0
  33. 2026ICMAPE: In-Context Multiagent Pure Explorationcited by 0
  34. 2026Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policiescited by 0