# Perceptive Locomotion through Nonlinear Model Predictive Control

arXiv: https://arxiv.org/abs/2208.08373 · HTML: https://mlsanity.com/paper/2208.08373/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2022.

## Built on by (most cited first, 30 of 38)

- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion](https://mlsanity.com/paper/2308.12517/) (2023)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [Learning Quadruped Locomotion Using Differentiable Simulation](https://mlsanity.com/paper/2403.14864/) (2024)
- [RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion](https://mlsanity.com/paper/2305.17842/) (2023)
- [Learning Visual Quadrupedal Loco-Manipulation from Demonstrations](https://mlsanity.com/paper/2403.20328/) (2024)
- [Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion](https://mlsanity.com/paper/2212.14400/) (2022)
- [Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllers](https://mlsanity.com/paper/2405.12424/) (2024)
- [Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids](https://mlsanity.com/paper/2601.07718/) (2026)
- [A Unified and General Humanoid Whole-Body Controller for Versatile Locomotion](https://mlsanity.com/paper/2502.03206/) (2025)
- [Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning](https://mlsanity.com/paper/2502.17219/) (2025)
- [Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion](https://mlsanity.com/paper/2408.02662/) (2024)
- [Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator](https://mlsanity.com/paper/2412.03012/) (2024)
- [VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory](https://mlsanity.com/paper/2601.08665/) (2026)
- [PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion](https://mlsanity.com/paper/2409.09441/) (2024)
- [CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal Control](https://mlsanity.com/paper/2408.09662/) (2024)
- [MARG: MAstering Risky Gap Terrains for Legged Robots with Elevation Mapping](https://mlsanity.com/paper/2509.20036/) (2025)
- [Disturbance-Aware Adaptive Compensation in Hybrid Force-Position Locomotion Policy for Legged Robots](https://mlsanity.com/paper/2506.00472/) (2025)
- [Learning to Walk With Less: A Dyna-Style Approach to Quadrupedal Locomotion](https://mlsanity.com/paper/2509.06296/) (2025)
- [Latent Linear Quadratic Regulator for Robotic Control Tasks](https://mlsanity.com/paper/2407.11107/) (2024)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)
- [Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation](https://mlsanity.com/paper/2506.02206/) (2025)
- [Integrating Trajectory Optimization and Reinforcement Learning for Quadrupedal Jumping with Terrain-Adaptive Landing](https://mlsanity.com/paper/2509.12776/) (2025)
- [Learning Terrain-Specialized Policies for Adaptive Locomotion in Challenging Environments](https://mlsanity.com/paper/2509.20635/) (2025)
- [Attention-Based Neural-Augmented Kalman Filter for Legged Robot State Estimation](https://mlsanity.com/paper/2601.18569/) (2026)
- [GPO: Growing Policy Optimization for Legged Robot Locomotion and Whole-Body Control](https://mlsanity.com/paper/2601.20668/) (2026)
- [Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking](https://mlsanity.com/paper/2606.08253/) (2026)
- [MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds](https://mlsanity.com/paper/2606.10288/) (2026)
- [Agile perceptive multi-skill locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2607.13579/) (2026)

## Abstract (arXiv)

Dynamic locomotion in rough terrain requires accurate foot placement, collision avoidance, and planning of the underactuated dynamics of the system. Reliably optimizing for such motions and interactions in the presence of imperfect and often incomplete perceptive information is challenging. We present a complete perception, planning, and control pipeline, that can optimize motions for all degrees of freedom of the robot in real-time. To mitigate the numerical challenges posed by the terrain a sequence of convex inequality constraints is extracted as local approximations of foothold feasibility and embedded into an online model predictive controller. Steppability classification, plane segmentation, and a signed distance field are precomputed per elevation map to minimize the computational effort during the optimization. A combination of multiple-shooting, real-time iteration, and a filter-based line-search are used to solve the formulated problem reliably and at high rate. We validate the proposed method in scenarios with gaps, slopes, and stepping stones in simulation and experimentally on the ANYmal quadruped platform, resulting in state-of-the-art dynamic climbing.