# Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams

arXiv: https://arxiv.org/abs/2505.13834 · HTML: https://mlsanity.com/paper/2505.13834/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Multi-agent reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- Deep reinforcement learning methods / [Multi-agent reinforcement learning](https://mlsanity.com/deeprl/multi-agent-reinforcement-learning/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Extreme Parkour with Legged Robots](https://mlsanity.com/paper/2309.14341/) (2023)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [From Motor Control to Team Play in Simulated Humanoid Football](https://mlsanity.com/paper/2105.12196/) (2021)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots](https://mlsanity.com/paper/2405.01792/) (2024)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot](https://mlsanity.com/paper/2208.01160/) (2022)
- [Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning](https://mlsanity.com/paper/2210.04435/) (2022)
- [Learning Visual Quadrupedal Loco-Manipulation from Demonstrations](https://mlsanity.com/paper/2403.20328/) (2024)
- [Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.02425/) (2024)
- [Robust Robot Walker: Learning Agile Locomotion over Tiny Traps](https://mlsanity.com/paper/2409.07409/) (2024)
- [MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.20326/) (2024)
- [Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer](https://mlsanity.com/paper/2412.09417/) (2024)
- [Designing a skilled soccer team for RoboCup: exploring skill-set-primitives through reinforcement learning](https://mlsanity.com/paper/2312.14360/) (2023)
- [Deep Reinforcement Learning to Acquire Navigation Skills for Wheel-Legged Robots in Complex Environments](https://mlsanity.com/paper/1804.10500/) (2018)
- [Two-stage training algorithm for AI robot soccer](https://mlsanity.com/paper/2104.05931/) (2021)
- [DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision](https://mlsanity.com/paper/2403.14300/) (2024)

## Built on by (most cited first, 3 of 3)

- [Humanoid Goalkeeper: Learning from Position Conditioned Task-Motion Constraints](https://mlsanity.com/paper/2510.18002/) (2025)
- [CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks](https://mlsanity.com/paper/2509.14380/) (2025)
- [RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learning](https://mlsanity.com/paper/2606.11092/) (2026)

## Abstract (arXiv)

Achieving coordinated teamwork among legged robots requires both fine-grained locomotion control and long-horizon strategic decision-making. Robot soccer offers a compelling testbed for this challenge, combining dynamic, competitive, and multi-agent interactions. In this work, we present a hierarchical multi-agent reinforcement learning (MARL) framework that enables fully autonomous and decentralized quadruped robot soccer. First, a set of highly dynamic low-level skills is trained for legged locomotion and ball manipulation, such as walking, dribbling, and kicking. On top of these, a high-level strategic planning policy is trained with Multi-Agent Proximal Policy Optimization (MAPPO) via Fictitious Self-Play (FSP). This learning framework allows agents to adapt to diverse opponent strategies and gives rise to sophisticated team behaviors, including coordinated passing, interception, and dynamic role allocation. With an extensive ablation study, the proposed learning method shows significant advantages in the cooperative and competitive multi-agent soccer game. We deploy the learned policies to real quadruped robots relying solely on onboard proprioception and decentralized localization, with the resulting system supporting autonomous robot-robot and robot-human soccer matches on indoor and outdoor soccer courts.