# First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion

arXiv: https://arxiv.org/abs/2007.01520 · HTML: https://mlsanity.com/paper/2007.01520/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [Reaching Through Latent Space: From Joint Statistics to Path Planning in Manipulation](https://mlsanity.com/paper/2210.11779/) (2022)
- [Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion](https://mlsanity.com/paper/2405.19452/) (2024)
- [Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions](https://mlsanity.com/paper/2106.04165/) (2021)
- [Leveraging Scene Embeddings for Gradient-Based Motion Planning in Latent Space](https://mlsanity.com/paper/2303.03364/) (2023)
- [Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion](https://mlsanity.com/paper/2112.04809/) (2021)
- [VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation](https://mlsanity.com/paper/2205.01179/) (2022)
- [Learning to Navigate from Scratch using World Models and Curiosity: the Good, the Bad, and the Ugly](https://mlsanity.com/paper/2308.15852/) (2023)

## Abstract (arXiv)

Traditional approaches to quadruped control frequently employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In addition, kinodynamic constraints are often non-differentiable and difficult to implement in an optimisation approach. In this work, these challenges are addressed by framing quadruped control as optimisation in a structured latent space. A deep generative model captures a statistical representation of feasible joint configurations, whilst complex dynamic and terminal constraints are expressed via high-level, semantic indicators and represented by learned classifiers operating upon the latent space. As a consequence, complex constraints are rendered differentiable and evaluated an order of magnitude faster than analytical approaches. We validate the feasibility of locomotion trajectories optimised using our approach both in simulation and on a real-world ANYmal quadruped. Our results demonstrate that this approach is capable of generating smooth and realisable trajectories. To the best of our knowledge, this is the first time latent space control has been successfully applied to a complex, real robot platform.