# Designing a skilled soccer team for RoboCup: exploring skill-set-primitives through reinforcement learning

arXiv: https://arxiv.org/abs/2312.14360 · HTML: https://mlsanity.com/paper/2312.14360/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2023.

## Built on by (most cited first, 6 of 6)

- [Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer](https://mlsanity.com/paper/2412.09417/) (2024)
- [Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams](https://mlsanity.com/paper/2505.13834/) (2025)
- [Enhancing Q-Learning with Large Language Model Heuristics](https://mlsanity.com/paper/2405.03341/) (2024)
- [RobocupGym: A challenging continuous control benchmark in Robocup](https://mlsanity.com/paper/2407.14516/) (2024)
- [HierKick: Hierarchical Reinforcement Learning for Vision-Guided Soccer Robot Control](https://mlsanity.com/paper/2603.00948/) (2026)
- [Reinforcement Learning Enabled Adaptive Multi-Task Control for Bipedal Soccer Robots](https://mlsanity.com/paper/2604.19104/) (2026)

## Abstract (arXiv)

The RoboCup 3D Soccer Simulation League serves as a competitive platform for showcasing innovation in autonomous humanoid robot agents through simulated soccer matches. Our team, FC Portugal, developed a new codebase from scratch in Python after RoboCup 2021. The team's performance relies on a set of skills centered around novel unifying primitives and a custom, symmetry-extended version of the Proximal Policy Optimization algorithm. Our methods have been thoroughly tested in official RoboCup matches, where FC Portugal has won the last two main competitions, in 2022 and 2023. This paper presents our training framework, as well as a timeline of skills developed using our skill-set-primitives, which considerably improve the sample efficiency and stability of skills, and motivate seamless transitions. We start with a significantly fast sprint-kick developed in 2021 and progress to the most recent skill set, including a multi-purpose omnidirectional walk, a dribble with unprecedented ball control, a solid kick, and a push skill. The push addresses low-level collision scenarios and high-level strategies to increase ball possession. We address the resource-intensive nature of this task through an innovative multi-agent learning approach. Finally, we release the team's codebase to the RoboCup community, providing other teams with a robust and modern foundation upon which they can build new features.