# Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response

arXiv: https://arxiv.org/abs/2312.11460 · HTML: https://mlsanity.com/paper/2312.11460/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 47 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 30 of 34)

- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions](https://mlsanity.com/paper/2203.15103/) (2022)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations](https://mlsanity.com/paper/2110.14565/) (2021)
- [Multi-expert learning of adaptive legged locomotion](https://mlsanity.com/paper/2012.05810/) (2020)
- [DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10602/) (2023)
- [Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion](https://mlsanity.com/paper/2011.02404/) (2020)
- [Neural Volumetric Memory for Visual Locomotion Control](https://mlsanity.com/paper/2304.01201/) (2023)
- [Elevation Mapping for Locomotion and Navigation using GPU](https://mlsanity.com/paper/2204.12876/) (2022)
- [Barkour: Benchmarking Animal-level Agility with Quadruped Robots](https://mlsanity.com/paper/2305.14654/) (2023)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion](https://mlsanity.com/paper/2303.11330/) (2023)

## Built on by (most cited first, 30 of 39)

- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit](https://mlsanity.com/paper/2502.13013/) (2025)
- [GRUtopia: Dream General Robots in a City at Scale](https://mlsanity.com/paper/2407.10943/) (2024)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion](https://mlsanity.com/paper/2405.10830/) (2024)
- [Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparities](https://mlsanity.com/paper/2507.13019/) (2025)
- [Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching](https://mlsanity.com/paper/2602.15827/) (2026)
- [World Model-based Perception for Visual Legged Locomotion](https://mlsanity.com/paper/2409.16784/) (2024)
- [Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation](https://mlsanity.com/paper/2502.10894/) (2025)
- [Robust Robot Walker: Learning Agile Locomotion over Tiny Traps](https://mlsanity.com/paper/2409.07409/) (2024)
- [PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion](https://mlsanity.com/paper/2409.09441/) (2024)
- [SLR: Learning Quadruped Locomotion without Privileged Information](https://mlsanity.com/paper/2406.04835/) (2024)
- [TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion](https://mlsanity.com/paper/2503.20839/) (2025)
- [GRAM: Generalization in Deep RL with a Robust Adaptation Module](https://mlsanity.com/paper/2412.04323/) (2024)
- [Investigating Lagrangian Neural Networks for Infinite Horizon Planning in Quadrupedal Locomotion](https://mlsanity.com/paper/2506.16079/) (2025)
- [Bipedalism for Quadrupedal Robots: Versatile Loco-Manipulation through Risk-Adaptive Reinforcement Learning](https://mlsanity.com/paper/2507.20382/) (2025)
- [Contrastive Representation Learning for Robust Sim-to-Real Transfer of Adaptive Humanoid Locomotion](https://mlsanity.com/paper/2509.12858/) (2025)
- [FALCON: Actively Decoupled Visuomotor Policies for Loco-Manipulation with Foundation-Model-Based Coordination](https://mlsanity.com/paper/2512.04381/) (2025)
- [Neural Internal Model Control: Learning a Robust Control Policy via Predictive Error Feedback](https://mlsanity.com/paper/2411.13079/) (2024)
- [Transferable Latent-to-Latent Locomotion Policy for Efficient and Versatile Motion Control of Diverse Legged Robots](https://mlsanity.com/paper/2503.17626/) (2025)
- [CHyLL: Learning Continuous Neural Representations of Hybrid Systems](https://mlsanity.com/paper/2512.10117/) (2025)
- [Toward Reliable Sim-to-Real Predictability for MoE-based Robust Quadrupedal Locomotion](https://mlsanity.com/paper/2602.00678/) (2026)
- [Agent-Driven Autonomous Reinforcement Learning Research: Iterative Policy Improvement for Quadruped Locomotion](https://mlsanity.com/paper/2603.27416/) (2026)
- [MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotion](https://mlsanity.com/paper/2505.00488/) (2025)
- [GRoQ-LoCO: Generalist and Robot-agnostic Quadruped Locomotion Control using Offline Datasets](https://mlsanity.com/paper/2505.10973/) (2025)
- [Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking](https://mlsanity.com/paper/2606.08253/) (2026)
- [ATRos: Learning Energy-Efficient Agile Locomotion for Wheeled-legged Robots](https://mlsanity.com/paper/2510.09980/) (2025)
- [CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrains](https://mlsanity.com/paper/2602.03511/) (2026)
- [Data-Driven Physics Embedded Dynamics with Predictive Control and Reinforcement Learning for Quadrupeds](https://mlsanity.com/paper/2603.14333/) (2026)

## Abstract (arXiv)

Robust locomotion control depends on accurate state estimations. However, the sensors of most legged robots can only provide partial and noisy observations, making the estimation particularly challenging, especially for external states like terrain frictions and elevation maps. Inspired by the classical Internal Model Control principle, we consider these external states as disturbances and introduce Hybrid Internal Model (HIM) to estimate them according to the response of the robot. The response, which we refer to as the hybrid internal embedding, contains the robot's explicit velocity and implicit stability representation, corresponding to two primary goals for locomotion tasks: explicitly tracking velocity and implicitly maintaining stability. We use contrastive learning to optimize the embedding to be close to the robot's successor state, in which the response is naturally embedded. HIM has several appealing benefits: It only needs the robot's proprioceptions, i.e., those from joint encoders and IMU as observations. It innovatively maintains consistent observations between simulation reference and reality that avoids information loss in mimicking learning. It exploits batch-level information that is more robust to noises and keeps better sample efficiency. It only requires 1 hour of training on an RTX 4090 to enable a quadruped robot to traverse any terrain under any disturbances. A wealth of real-world experiments demonstrates its agility, even in high-difficulty tasks and cases never occurred during the training process, revealing remarkable open-world generalizability.