# MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2409.20326 · HTML: https://mlsanity.com/paper/2409.20326/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2024.

## Built on by (most cited first, 7 of 7)

- [RSL-RL: A Learning Library for Robotics Research](https://mlsanity.com/paper/2509.10771/) (2025)
- [Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.01522/) (2025)
- [Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams](https://mlsanity.com/paper/2505.13834/) (2025)
- [CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks](https://mlsanity.com/paper/2509.14380/) (2025)
- [Generative World Models of Tasks: LLM-Driven Hierarchical Scaffolding for Embodied Agents](https://mlsanity.com/paper/2509.04731/) (2025)
- [HARL-A: An Extensible Benchmark Framework for Heterogeneous Multi-Agent Adversarial Reinforcement Learning in IsaacLab](https://mlsanity.com/paper/2510.01264/) (2025)
- [Enhance the Safety in Reinforcement Learning by ADRC Lagrangian Methods](https://mlsanity.com/paper/2601.18142/) (2026)

## Abstract (arXiv)

Robot soccer, in its full complexity, poses an unsolved research challenge. Current solutions heavily rely on engineered heuristic strategies, which lack robustness and adaptability. Deep reinforcement learning has gained significant traction in various complex robotics tasks such as locomotion, manipulation, and competitive games (e.g., AlphaZero, OpenAI Five), making it a promising solution to the robot soccer problem. This paper introduces MARLadona. A decentralized multi-agent reinforcement learning (MARL) training pipeline capable of producing agents with sophisticated team play behavior, bridging the shortcomings of heuristic methods. Furthermore, we created an open-source multi-agent soccer environment. Utilizing our MARL framework and a modified global entity encoder (GEE) as our core architecture, our approach achieves a 66.8% win rate against HELIOS agent, which employs a state-of-the-art heuristic strategy. In addition, we provided an in-depth analysis of the policy behavior and interpreted the agent's intention using the critic network.