mlsanityData to 14 Oct 2026

Training Generalizable Collaborative Agents via Strategic Risk Aversion

arXiv 2602.21515 ↗First posted 25 Feb 2026cs.LG cs.AI cs.MAcited by 2

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Many emerging agentic paradigms require agents to collaborate with one another (or people) to achieve shared goals. Unfortunately, existing approaches to learning policies for such collaborative problems produce brittle solutions that fail when paired with new partners. We attribute these failures to a combination of free-riding during training and a lack of strategic robustness. To address these problems, we study the concept of strategic risk aversion and interpret it as a principled inductive bias for generalizable cooperation with unseen partners. While strategically risk-averse players are robust to deviations in their partner's behavior by design, we show that, in collaborative games, they also (1) can have better equilibrium outcomes than those at classical game-theoretic concepts like Nash, and (2) exhibit less or no free-riding. Inspired by these insights, we develop a multi-agent reinforcement learning (MARL) algorithm that integrates strategic risk aversion into standard policy optimization methods. Our empirical results across collaborative benchmarks (including an LLM collaboration task) validate our theory and demonstrate that our approach consistently achieves reliable collaboration with heterogeneous and previously unseen partners across collaborative tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 68 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

47 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning25 Feb

Other references · 46

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2015Trust Region Policy Optimizationcited by 2,580
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  10. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  11. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  12. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
34 more
  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  2. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  3. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  4. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 220
  5. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  6. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  7. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  8. 2018Understanding the impact of entropy on policy optimizationcited by 176
  9. 2021Policy Mirror Descent for Reinforcement Learning: Linear Convergence, New Sampling Complexity, and Generalized Problem Classescited by 156
  10. 2021Off-Belief Learningcited by 75
  11. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  12. 2019Reinforcement Learning for Market Making in a Multi-agent Dealer Marketcited by 39
  13. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  14. 2021A New Formalism, Method and Open Issues for Zero-Shot Coordinationcited by 33
  15. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  16. 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25
  17. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 25
  18. 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 24
  19. 2024Learning to Cooperate with Humans using Generative Agentscited by 24
  20. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 20
  21. 2022First-order Conditions for Optimization in the Wasserstein Spacecited by 19
  22. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  23. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  24. 2022A Game-Theoretic Framework for Managing Risk in Multi-Agent Systemscited by 14
  25. 2025Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberationcited by 11
  26. 2025Ad-Hoc Human-AI Coordination Challengecited by 8
  27. 2024Variational Analysis in the Wasserstein Spacecited by 6
  28. 2022Risk-Sensitive Bayesian Games for Multi-Agent Reinforcement Learning under Policy Uncertaintycited by 5
  29. 2020Risk-Averse Equilibrium for Gamescited by 4
  30. 2024Tractable Equilibrium Computation in Markov Games through Risk Aversioncited by 4
  31. 2025Strategically Robust Game Theory via Optimal Transportcited by 4
  32. 2017Nash and Wardrop equilibria in aggregative games with coupling constraintscited by 3
  33. 2025Convergent Q-Learning for Infinite-Horizon General-Sum Markov Games through Behavioral Economicscited by 2
  34. 2025Robust and Diverse Multi-Agent Learning via Rational Policy Gradientcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Role Steering of Language Models for Social Simulationscited by 1
  2. 2026Symbolic Reasoning Frameworks Trigger Memory-Mediated Ecosystem Dynamics in Multi-Agent LLM Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Price of Paranoia: Robust Risk-Sensitive Cooperation in Non-Stationary Multi-Agent Reinforcement Learning
  2. 2024Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies
  3. 2024On the Complexity of Learning to Cooperate with Populations of Socially Rational Agents
  4. 2025Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
  5. 2024Multi-agent cooperation through learning-aware policy gradients
  6. 2025LLM Collaboration With Multi-Agent Reinforcement Learning
  7. 2024Emergent Cooperation under Uncertain Incentive Alignment
  8. 2025Unsupervised Partner Design Enables Robust Ad-hoc Teamwork