mlsanityData to 14 Oct 2026

Multiagent Soft Q-Learning

arXiv 1804.09817 ↗First posted 25 Apr 2018cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Policy gradient methods are often applied to reinforcement learning in continuous multiagent games. These methods perform local search in the joint-action space, and as we show, they are susceptable to a game-theoretic pathology known as relative overgeneralization. To resolve this issue, we propose Multiagent Soft Q-learning, which can be seen as the analogue of applying Q-learning to continuous controls. We compare our method to MADDPG, a state-of-the-art approach, and show that our method achieves better coordination in multiagent cooperative tasks, converging to better local optima in the joint action space.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 11; 2023: 3; 2024: 3; 2025: 2; 2026: 1.

26 papers.

Sort:

By area:

  1. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  2. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  3. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
  4. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  5. 2021Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rolloutscited by 24
  6. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  7. 2022Multi-Agent Coordination via Multi-Level Communicationcited by 14
  8. 2021Emergence of Theory of Mind Collaboration in Multiagent Systemscited by 8
  9. 2022Greedy based Value Representation for Optimal Coordination in Multi-agent Reinforcement Learningcited by 4
  10. 2021Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarioscited by 3
  11. 2022Multi-agent Actor-Critic with Time Dynamical Opponent Modelcited by 3
  12. 2022Residual Q-Networks for Value Function Factorizing in Multi-Agent Reinforcement Learningcited by 3
14 more
  1. 2022Learning Explicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning via Polarization Policy Gradientcited by 3
  2. 2024Finite-Time Error Analysis of Soft Q-Learning: Switching System Approachcited by 3
  3. 2025Multi-level Advantage Credit Assignment for Cooperative Multi-Agent Reinforcement Learningcited by 3
  4. 2022Recursive Reasoning Graph for Multi-Agent Reinforcement Learningcited by 2
  5. 2022Decision-making with Speculative Opponent Modelscited by 2
  6. 2022CURO: Curriculum Learning for Relative Overgeneralizationcited by 1
  7. 2023Negotiated Reasoning: On Provably Addressing Relative Over-Generalizationcited by 1
  8. 2023Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jetscited by 1
  9. 2024Joint Intrinsic Motivation for Coordinated Exploration in Multi-Agent Deep Reinforcement Learningcited by 1
  10. 2022PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learningcited by 0
  11. 2022Knowing the Past to Predict the Future: Reinforcement Virtual Learningcited by 0
  12. 2024Mitigating Relative Over-Generalization in Multi-Agent Reinforcement Learningcited by 0
  13. 2025Vairiational Stochastic Gamescited by 0
  14. 2026Multi-Agent Decision-Focused Learning via Value-Aware Sequential Communicationcited by 0