mlsanityData to 14 Oct 2026

QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning

arXiv 1905.05408 ↗First posted 14 May 2019cs.LG cs.AI cs.MAcited by 379

Not on the map. Why?

Cited by 22 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We explore value-based solutions for multi-agent reinforcement learning (MARL) tasks in the centralized training with decentralized execution (CTDE) regime popularized recently. However, VDN and QMIX are representative examples that use the idea of factorization of the joint action-value function into individual ones for decentralized execution. VDN and QMIX address only a fraction of factorizable MARL tasks due to their structural constraint in factorization such as additivity and monotonicity. In this paper, we propose a new factorization method for MARL, QTRAN, which is free from such structural constraints and takes on a new approach to transforming the original joint action-value function into an easily factorizable one, with the same optimal actions. QTRAN guarantees more general factorization than VDN or QMIX, thus covering a much wider class of MARL tasks than does previous methods. Our experiments for the tasks of multi-domain Gaussian-squeeze and modified predator-prey demonstrate QTRAN's superior performance with especially larger margins in games whose payoffs penalize non-cooperative behavior more aggressively.

Key facts. Cited by 379 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 60; 2022: 75; 2023: 64; 2024: 57; 2025: 72; 2026: 51.

379 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  2. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  3. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  4. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 114
  5. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 104
  6. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 98
  7. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 79
  8. 2024Simplifying Deep Temporal Difference Learningcited by 78
  9. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  10. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  11. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 56
  12. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
38 more
  1. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  2. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  3. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 48
  4. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  5. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 43
  6. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  7. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 42
  8. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 38
  9. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 37
  10. 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 36
  11. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  12. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 35
  13. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  14. 2021Offline Decentralized Multi-Agent Reinforcement Learningcited by 33
  15. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  16. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  17. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 32
  18. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Taskscited by 32
  19. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  20. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  21. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 30
  22. 2023A Review of Cooperation in Multi-agent Learningcited by 29
  23. 2021Context-Aware Sparse Deep Coordination Graphscited by 28
  24. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  25. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
  26. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  27. 2022Scalable Multi-Agent Model-Based Reinforcement Learningcited by 25
  28. 2022Solving Continuous Control via Q-learningcited by 24
  29. 2025Embodied AI: From LLMs to World Modelscited by 24
  30. 2022Quantum Multi-Agent Meta Reinforcement Learningcited by 23
  31. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 20
  32. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  33. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 20
  34. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  35. 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
  36. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  37. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  38. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19

50 most cited of 379 All 379 (JSON)