# Spatiotemporal Agility: Time-Constrained Reinforcement Learning for Vision-Guided Dynamic Quadrupedal Interception

arXiv: https://arxiv.org/abs/2608.06907 · HTML: https://mlsanity.com/paper/2608.06907/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 26 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Extreme Parkour with Legged Robots](https://mlsanity.com/paper/2309.14341/) (2023)
- [ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots](https://mlsanity.com/paper/2306.14874/) (2023)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [YOLOv13: Real-Time Object Detection with Hypergraph-Enhanced Adaptive Visual Perception](https://mlsanity.com/paper/2506.17733/) (2025)
- [Advanced Skills by Learning Locomotion and Local Navigation End-to-End](https://mlsanity.com/paper/2209.12827/) (2022)
- [CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion](https://mlsanity.com/paper/2405.10830/) (2024)
- [Event-based Agile Object Catching with a Quadrupedal Robot](https://mlsanity.com/paper/2303.17479/) (2023)
- [Learning Agile Locomotion on Risky Terrains](https://mlsanity.com/paper/2311.10484/) (2023)
- [Catch It! Learning to Catch in Flight with Mobile Dexterous Hands](https://mlsanity.com/paper/2409.10319/) (2024)
- [Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator](https://mlsanity.com/paper/2412.03012/) (2024)
- [Catch the Ball: Accurate High-Speed Motions for Mobile Manipulators via Inverse Dynamics Learning](https://mlsanity.com/paper/2003.07489/) (2020)
- [Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning](https://mlsanity.com/paper/2310.15583/) (2023)
- [Perception for Humanoid Robots](https://mlsanity.com/paper/2309.15616/) (2023)
- [Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobility](https://mlsanity.com/paper/2505.16084/) (2025)
- [Neural Motion Prediction for In-flight Uneven Object Catching](https://mlsanity.com/paper/2103.08368/) (2021)
- [Dynamic Object Catching with Quadruped Robot Front Legs](https://mlsanity.com/paper/2410.08065/) (2024)
- [Beyond Robustness: Learning Unknown Dynamic Load Adaptation for Quadruped Locomotion on Rough Terrain](https://mlsanity.com/paper/2507.07825/) (2025)
- [Impedance Matching: Enabling an RL-Based Running Jump in a Quadruped Robot](https://mlsanity.com/paper/2404.15096/) (2024)
- [Learning Diverse Natural Behaviors for Enhancing the Agility of Quadrupedal Robots](https://mlsanity.com/paper/2505.09979/) (2025)

## Abstract (arXiv)

Legged robots require robust agility to perceive and interact with complex and dynamic environments within a constrained time. However, most existing quadruped locomotion works rely on velocity-tracking policy, which struggle to reach precise targets within strict temporal constraints. Moreover, integrating real-time perception with agile locomotion for highly dynamic targets remains challenging due to sensor latency and processing delays. To concretely study and benchmark such agility in dynamic settings, we introduce a challenging ball-catching task for legged robots. This paper proposes an integrated framework that combines a vision module for landing point and time prediction with a direct position and time conditioned RL locomotion policy, instead of intermediate velocity commands. Beyond the method design, this work presents a system-level contribution that completes real-time robotic interception system that integrates multi-camera perception, online trajectory prediction, low-latency target communication, and sim-to-real locomotion control into a closed-loop deployment pipeline. By explicitly predicting the future spatial-temporal target, our approach mitigates perception latency during dynamic interception. We conducted extensive ball-catching experiments for the legged robot. Through comparative experiments against a velocity-tracking baseline, our direct target-conditioned approach achieves a higher success rate in catching balls with predicted landing spots within 2 meters and flight times between 0.8 and 1.2 seconds. This shows that the robot has successfully completed the dynamic ball-catching task under our tested setup. Furthermore, our policy exhibits a smaller performance gap after deployment, suggesting improved sim-to-real behavior in these trials.