mlsanityData to 14 Oct 2026

Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot

arXiv 2208.01160 ↗First posted 1 Aug 2022cs.RO cs.AI cs.SYcited by 38

Not on the map. Why?

Abstract (arXiv)

We address the problem of enabling quadrupedal robots to perform precise shooting skills in the real world using reinforcement learning. Developing algorithms to enable a legged robot to shoot a soccer ball to a given target is a challenging problem that combines robot motion control and planning into one task. To solve this problem, we need to consider the dynamics limitation and motion stability during the control of a dynamic legged robot. Moreover, we need to consider motion planning to shoot the hard-to-model deformable ball rolling on the ground with uncertain friction to a desired location. In this paper, we propose a hierarchical framework that leverages deep reinforcement learning to train (a) a robust motion control policy that can track arbitrary motions and (b) a planning policy to decide the desired kicking motion to shoot a soccer ball to a target. We deploy the proposed framework on an A1 quadrupedal robot and enable it to accurately shoot the ball to random targets in the real world.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Builds on 16 of its 36 references that are in the map. First posted on arXiv 1 Aug 2022.

Builds on

16 of 36 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  3. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  4. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  5. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  6. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  7. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  8. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  9. 2022Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotioncited by 88
  10. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
  11. 2019Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Controlcited by 54
  12. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
4 more
  1. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 32
  2. 2017Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learningcited by 27
  3. 2020Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbscited by 16
  4. 2021Autonomous Navigation for Quadrupedal Robots with Optimized Jumping through Constrained Obstaclescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 10; 2025: 12; 2026: 2.

38 papers.

Sort:

By area:

  1. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  2. 2023Robot Parkour Learningcited by 149
  3. 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
  4. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  5. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  6. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 48
  7. 2023DribbleBot: Dynamic Legged Manipulation in the Wildcited by 40
  8. 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 39
  9. 2023Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Modelscited by 37
  10. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 36
  11. 2023Prompt, Plan, Perform: LLM-based Humanoid Control via Quantized Imitation Learningcited by 31
  12. 2024Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policiescited by 30
26 more
  1. 2024Pedipulate: Enabling Manipulation Skills using a Quadruped Robot's Legcited by 27
  2. 2022Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learningcited by 23
  3. 2024Learning Visual Quadrupedal Loco-Manipulation from Demonstrationscited by 20
  4. 2025Bridging the Sim-to-Real Gap for Athletic Loco-Manipulationcited by 13
  5. 2025SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blendingcited by 9
  6. 2025Humanoid Goalkeeper: Learning from Position Conditioned Task-Motion Constraintscited by 9
  7. 2024PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robotscited by 8
  8. 2024Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccercited by 7
  9. 2024HiLMa-Res: A General Hierarchical Framework via Residual RL for Combining Quadrupedal Locomotion and Manipulationcited by 4
  10. 2025VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Playcited by 4
  11. 2024DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervisioncited by 3
  12. 2025Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teamscited by 3
  13. 2025Bipedalism for Quadrupedal Robots: Versatile Loco-Manipulation through Risk-Adaptive Reinforcement Learningcited by 3
  14. 2023Facilitating Sim-to-real by Intrinsic Stochasticity of Real-Time Simulation in Reinforcement Learning for Robot Manipulationcited by 2
  15. 2025AURA: Autonomous Upskilling with Retrieval-Augmented Agentscited by 2
  16. 2022Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemblecited by 1
  17. 2023Efficient Reinforcement Learning for Jumping Monopodscited by 1
  18. 2023A Multifidelity Sim-to-Real Pipeline for Verifiable and Compositional Reinforcement Learningcited by 0
  19. 2023MASP: Scalable GNN-based Planning for Multi-Agent Navigationcited by 0
  20. 2025CAIMAN: Causal Action Influence Detection for Sample-efficient Loco-manipulationcited by 0
  21. 2025Coordinating Spinal and Limb Dynamics for Enhanced Sprawling Robot Mobilitycited by 0
  22. 2025Dynamic Legged Ball Manipulation on Rugged Terrains with Hierarchical Reinforcement Learningcited by 0
  23. 2025Scoop-and-Toss: Dynamic Object Collection for Quadrupedal Systemscited by 0
  24. 2025GeCCo -- a Generalist Contact-Conditioned Policy for Loco-Manipulation Skills on Legged Robotscited by 0
  25. 2026CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrainscited by 0
  26. 2026GR2PO: Group Relative Return Policy Optimization for Continuous Robot Controlcited by 0