# From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion

arXiv: https://arxiv.org/abs/2011.11722 · HTML: https://mlsanity.com/paper/2011.11722/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2020.

## Built on by (most cited first, 20 of 20)

- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots](https://mlsanity.com/paper/2405.01792/) (2024)
- [Learning to Jump from Pixels](https://mlsanity.com/paper/2110.15344/) (2021)
- [Neural Volumetric Memory for Visual Locomotion Control](https://mlsanity.com/paper/2304.01201/) (2023)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization](https://mlsanity.com/paper/2109.14549/) (2021)
- [ViNL: Visual Navigation and Locomotion Over Obstacles](https://mlsanity.com/paper/2210.14791/) (2022)
- [Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior](https://mlsanity.com/paper/2310.01408/) (2023)
- [Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity](https://mlsanity.com/paper/2210.09918/) (2022)
- [Hierarchical Quality-Diversity for Online Damage Recovery](https://mlsanity.com/paper/2204.05726/) (2022)
- [Implicit Two-Tower Policies](https://mlsanity.com/paper/2208.01191/) (2022)
- [A Transferable Legged Mobile Manipulation Framework Based on Disturbance Predictive Control](https://mlsanity.com/paper/2203.03391/) (2022)
- [Zeroth-Order Actor-Critic: An Evolutionary Framework for Sequential Decision Problems](https://mlsanity.com/paper/2201.12518/) (2022)
- [Ancestral Reinforcement Learning: Unifying Zeroth-Order Optimization and Genetic Algorithms for Reinforcement Learning](https://mlsanity.com/paper/2408.09493/) (2024)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)
- [Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning](https://mlsanity.com/paper/2209.04854/) (2022)
- [Arrive and Survive: Scaling Safe Goal-Conditioned Policy Learning from One-Bit Failure Signals](https://mlsanity.com/paper/2608.26571/) (2026)

## Abstract (arXiv)

Legged robots navigating crowded scenes and complex terrains in the real world are required to execute dynamic leg movements while processing visual input for obstacle avoidance and path planning. We show that a quadruped robot can acquire both of these skills by means of hierarchical reinforcement learning (HRL). By virtue of their hierarchical structure, our policies learn to implicitly break down this joint problem by concurrently learning High Level (HL) and Low Level (LL) neural network policies. These two levels are connected by a low dimensional hidden layer, which we call latent command. HL receives a first-person camera view, whereas LL receives the latent command from HL and the robot's on-board sensors to control its actuators. We train policies to walk in two different environments: a curved cliff and a maze. We show that hierarchical policies can concurrently learn to locomote and navigate in these environments, and show they are more efficient than non-hierarchical neural network policies. This architecture also allows for knowledge reuse across tasks. LL networks trained on one task can be transferred to a new task in a new environment. Finally HL, which processes camera images, can be evaluated at much lower and varying frequencies compared to LL, thus reducing computation times and bandwidth requirements.