# DTC: Deep Tracking Control

arXiv: https://arxiv.org/abs/2309.15462 · HTML: https://mlsanity.com/paper/2309.15462/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control](https://mlsanity.com/paper/2012.03094/) (2020)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [Advanced Skills by Learning Locomotion and Local Navigation End-to-End](https://mlsanity.com/paper/2209.12827/) (2022)
- [DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning](https://mlsanity.com/paper/1909.08399/) (2019)
- [Elevation Mapping for Locomotion and Navigation using GPU](https://mlsanity.com/paper/2204.12876/) (2022)
- [Perceptive Locomotion through Nonlinear Model Predictive Control](https://mlsanity.com/paper/2208.08373/) (2022)
- [Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors](https://mlsanity.com/paper/2203.17138/) (2022)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimization](https://mlsanity.com/paper/2107.06629/) (2021)
- [TAMOLS: Terrain-Aware Motion Optimization for Legged Systems](https://mlsanity.com/paper/2206.14049/) (2022)
- [Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking](https://mlsanity.com/paper/2203.07589/) (2022)
- [Motion Planning for Quadrupedal Locomotion: Coupled Planning, Terrain Mapping and Whole-Body Control](https://mlsanity.com/paper/2003.05481/) (2020)
- [Receding-Horizon Perceptive Trajectory Optimization for Dynamic Legged Locomotion with Learned Initialization](https://mlsanity.com/paper/2104.09078/) (2021)
- [Footstep Planning for Autonomous Walking Over Rough Terrain](https://mlsanity.com/paper/1907.08673/) (2019)
- [Agile Maneuvers in Legged Robots: a Predictive Control Approach](https://mlsanity.com/paper/2203.07554/) (2022)
- [Reliable Trajectories for Dynamic Quadrupeds using Analytical Costs and Learned Initializations](https://mlsanity.com/paper/2002.06719/) (2020)
- [Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner](https://mlsanity.com/paper/2111.00262/) (2021)

## Built on by (most cited first, 30 of 56)

- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Learning Visual Quadrupedal Loco-Manipulation from Demonstrations](https://mlsanity.com/paper/2403.20328/) (2024)
- [Grasp Multiple Objects with One Hand](https://mlsanity.com/paper/2310.15599/) (2023)
- [DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human References](https://mlsanity.com/paper/2502.09614/) (2025)
- [Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimicking](https://mlsanity.com/paper/2412.17730/) (2024)
- [Learning Speed Adaptation for Flight in Clutter](https://mlsanity.com/paper/2403.04586/) (2024)
- [Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation](https://mlsanity.com/paper/2410.03076/) (2024)
- [Learning Humanoid Locomotion with World Model Reconstruction](https://mlsanity.com/paper/2502.16230/) (2025)
- [Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids](https://mlsanity.com/paper/2601.07718/) (2026)
- [A Unified and General Humanoid Whole-Body Controller for Versatile Locomotion](https://mlsanity.com/paper/2502.03206/) (2025)
- [ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Control](https://mlsanity.com/paper/2602.00401/) (2026)
- [Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion](https://mlsanity.com/paper/2408.02662/) (2024)
- [Identifying Terrain Physical Parameters from Vision -- Towards Physical-Parameter-Aware Locomotion and Navigation](https://mlsanity.com/paper/2408.16567/) (2024)
- [Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation](https://mlsanity.com/paper/2410.13817/) (2024)
- [Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator](https://mlsanity.com/paper/2412.03012/) (2024)
- [Distillation-PPO: A Novel Two-Stage Reinforcement Learning Framework for Humanoid Robot Perceptive Locomotion](https://mlsanity.com/paper/2503.08299/) (2025)
- [One Filter to Deploy Them All: Robust Safety for Quadrupedal Navigation in Unknown Environments](https://mlsanity.com/paper/2412.09989/) (2024)
- [Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining](https://mlsanity.com/paper/2506.16475/) (2025)
- [Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation](https://mlsanity.com/paper/2412.02676/) (2024)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Automatic Environment Shaping is the Next Frontier in RL](https://mlsanity.com/paper/2407.16186/) (2024)
- [CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal Control](https://mlsanity.com/paper/2408.09662/) (2024)
- [Adaptive Diffusion Terrain Generator for Autonomous Uneven Terrain Navigation](https://mlsanity.com/paper/2410.10766/) (2024)
- [Ego-Vision World Model for Humanoid Contact Planning](https://mlsanity.com/paper/2510.11682/) (2025)
- [MARG: MAstering Risky Gap Terrains for Legged Robots with Elevation Mapping](https://mlsanity.com/paper/2509.20036/) (2025)
- [DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision](https://mlsanity.com/paper/2403.14300/) (2024)
- [Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation](https://mlsanity.com/paper/2412.20338/) (2024)
- [No More Blind Spots: Learning Vision-Based Omnidirectional Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2508.11929/) (2025)

## Abstract (arXiv)

Legged locomotion is a complex control problem that requires both accuracy and robustness to cope with real-world challenges. Legged systems have traditionally been controlled using trajectory optimization with inverse dynamics. Such hierarchical model-based methods are appealing due to intuitive cost function tuning, accurate planning, generalization, and most importantly, the insightful understanding gained from more than one decade of extensive research. However, model mismatch and violation of assumptions are common sources of faulty operation. Simulation-based reinforcement learning, on the other hand, results in locomotion policies with unprecedented robustness and recovery skills. Yet, all learning algorithms struggle with sparse rewards emerging from environments where valid footholds are rare, such as gaps or stepping stones. In this work, we propose a hybrid control architecture that combines the advantages of both worlds to simultaneously achieve greater robustness, foot-placement accuracy, and terrain generalization. Our approach utilizes a model-based planner to roll out a reference motion during training. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We evaluate the accuracy of our locomotion pipeline on sparse terrains, where pure data-driven methods are prone to fail. Furthermore, we demonstrate superior robustness in the presence of slippery or deformable ground when compared to model-based counterparts. Finally, we show that our proposed tracking controller generalizes across different trajectory optimization methods not seen during training. In conclusion, our work unites the predictive capabilities and optimality guarantees of online planning with the inherent robustness attributed to offline learning.