mlsanityData to 7 Oct 2026

Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer

arXiv 2412.09417 ↗First posted 12 Dec 2024cs.RO cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Robot decision-making in partially observable, real-time, dynamic, and multi-agent environments remains a difficult and unsolved challenge. Model-free reinforcement learning (RL) is a promising approach to learning decision-making in such domains, however, end-to-end RL in complex environments is often intractable. To address this challenge in the RoboCup Standard Platform League (SPL) domain, we developed a novel architecture integrating RL within a classical robotics stack, while employing a multi-fidelity sim2real approach and decomposing behavior into learned sub-behaviors with heuristic selection. Our architecture led to victory in the 2024 RoboCup SPL Challenge Shield Division. In this work, we fully describe our system's architecture and empirically analyze key design decisions that contributed to its success. Our approach demonstrates how RL-based behaviors can be integrated into complete robot behavior architectures.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 26 of its 46 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

26 of 46 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2019Solving Rubik's Cube with a Robot Handcited by 699
  3. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  4. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  5. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 95
  6. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  7. 2019Emergent Coordination Through Competitioncited by 57
  8. 2019Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Realcited by 39
  9. 2023DribbleBot: Dynamic Legged Manipulation in the Wildcited by 33
  10. 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robotcited by 32
  11. 2023Learning Vision-Based Bipedal Locomotion for Challenging Terraincited by 30
  12. 2022Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigationcited by 28
14 more
  1. 2024Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physicscited by 24
  2. 2024Achieving Human Level Competitive Robot Table Tenniscited by 24
  3. 2020Learning Memory-Based Control for Human-Scale Bipedal Locomotioncited by 20
  4. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  5. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 18
  6. 2022Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learningcited by 17
  7. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 14
  8. 2019Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learningcited by 13
  9. 2020Planning in Learned Latent Action Spaces for Generalizable Legged Locomotioncited by 11
  10. 2018Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Bipedcited by 8
  11. 2022Multi-fidelity reinforcement learning framework for shape optimizationcited by 8
  12. 2023Designing a skilled soccer team for RoboCup: exploring skill-set-primitives through reinforcement learningcited by 5
  13. 2022Black-Box Safety Validation of Autonomous Systems: A Multi-Fidelity Reinforcement Learning Approachcited by 3
  14. 2024Multi-Fidelity Reinforcement Learning for Time-Optimal Quadrotor Re-planningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2026Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policycited by 3
  2. 2026GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learningcited by 2
  3. 2025Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teamscited by 1
  4. 2025Multi-Robot Collaboration through Reinforcement Learning and Abstract Simulationcited by 0
  5. 2026Reinforcement Learning Enabled Adaptive Multi-Task Control for Bipedal Soccer Robotscited by 0