# Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End

arXiv: https://arxiv.org/abs/2310.03581 · HTML: https://mlsanity.com/paper/2310.03581/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 58 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 30 of 37)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots](https://mlsanity.com/paper/2306.14874/) (2023)
- [Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning](https://mlsanity.com/paper/1611.03631/) (2016)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning](https://mlsanity.com/paper/1710.03937/) (2017)
- [Combining Optimal Control and Learning for Visual Navigation in Novel Environments](https://mlsanity.com/paper/1903.02531/) (2019)
- [Advanced Skills by Learning Locomotion and Local Navigation End-to-End](https://mlsanity.com/paper/2209.12827/) (2022)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Fast Traversability Estimation for Wild Visual Navigation](https://mlsanity.com/paper/2305.08510/) (2023)
- [Elevation Mapping for Locomotion and Navigation using GPU](https://mlsanity.com/paper/2204.12876/) (2022)
- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [Emergence of Maps in the Memories of Blind Navigation Agents](https://mlsanity.com/paper/2301.13261/) (2023)
- [Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigation](https://mlsanity.com/paper/2207.10821/) (2022)
- [Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments](https://mlsanity.com/paper/2203.15854/) (2022)
- [The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation](https://mlsanity.com/paper/2108.11550/) (2021)
- [Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning](https://mlsanity.com/paper/1706.09829/) (2017)
- [Learning Arm-Assisted Fall Damage Reduction and Recovery for Legged Mobile Manipulators](https://mlsanity.com/paper/2303.05486/) (2023)
- [Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors](https://mlsanity.com/paper/2012.00201/) (2020)
- [Proactive Anomaly Detection for Robot Navigation with Multi-Sensor Fusion](https://mlsanity.com/paper/2204.01146/) (2022)
- [An Efficient Locally Reactive Controller for Safe Navigation in Visual Teach and Repeat Missions](https://mlsanity.com/paper/2201.03938/) (2022)
- [Collision-Free MPC for Legged Robots in Static and Dynamic Scenes](https://mlsanity.com/paper/2103.13987/) (2021)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation; Results from the DARPA Subterranean Challenge](https://mlsanity.com/paper/2303.01614/) (2023)
- [IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors without any Outdoor Experience](https://mlsanity.com/paper/2305.01098/) (2023)
- [See Yourself in Others: Attending Multiple Tasks for Own Failure Detection](https://mlsanity.com/paper/2110.02549/) (2021)
- [ArtPlanner: Robust Legged Robot Navigation in the Field](https://mlsanity.com/paper/2303.01420/) (2023)

## Built on by (most cited first, 23 of 23)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [Spatially-Enhanced Recurrent Memory for Long-Range Mapless Navigation via End-to-End Reinforcement Learning](https://mlsanity.com/paper/2506.05997/) (2025)
- [Identifying Terrain Physical Parameters from Vision -- Towards Physical-Parameter-Aware Locomotion and Navigation](https://mlsanity.com/paper/2408.16567/) (2024)
- [Distillation-PPO: A Novel Two-Stage Reinforcement Learning Framework for Humanoid Robot Perceptive Locomotion](https://mlsanity.com/paper/2503.08299/) (2025)
- [One Filter to Deploy Them All: Robust Safety for Quadrupedal Navigation in Unknown Environments](https://mlsanity.com/paper/2412.09989/) (2024)
- [Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2410.07877/) (2024)
- [Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobility](https://mlsanity.com/paper/2505.16084/) (2025)
- [No More Blind Spots: Learning Vision-Based Omnidirectional Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2508.11929/) (2025)
- [DeFM: Learning Foundation Representations from Depth for Robotics](https://mlsanity.com/paper/2601.18923/) (2026)
- [HyperDet: 3D Object Detection with Hyper 4D Radar Point Clouds](https://mlsanity.com/paper/2602.11554/) (2026)
- [TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation](https://mlsanity.com/paper/2404.15256/) (2024)
- [Robust Localization, Mapping, and Navigation for Quadruped Robots](https://mlsanity.com/paper/2505.02272/) (2025)
- [VLD: Visual Language Goal Distance for Reinforcement Learning Navigation](https://mlsanity.com/paper/2512.07976/) (2025)
- [Vision-Language-Policy Model for Dynamic Robot Task Planning](https://mlsanity.com/paper/2512.19178/) (2025)
- [Toward Reliable Sim-to-Real Predictability for MoE-based Robust Quadrupedal Locomotion](https://mlsanity.com/paper/2602.00678/) (2026)
- [Prompt-responsive Object Retrieval with Memory-augmented Student-Teacher Learning](https://mlsanity.com/paper/2505.02232/) (2025)
- [Flexible Locomotion Learning with Diffusion Model Predictive Control](https://mlsanity.com/paper/2510.04234/) (2025)
- [TRANS: Terrain-aware Reinforcement Learning for Agile Navigation of Quadruped Robots under Social Interactions](https://mlsanity.com/paper/2602.12724/) (2026)
- [DreamFlow: Local Navigation Beyond Observation via Conditional Flow Matching in the Latent Space](https://mlsanity.com/paper/2603.02976/) (2026)
- [PALo: Learning Posture-Aware Locomotion for Quadruped Robots](https://mlsanity.com/paper/2503.04462/) (2025)
- [GUIDE: Goal-Initialized Directional Understanding for End-to-End Legged Navigation](https://mlsanity.com/paper/2606.10832/) (2026)
- [GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures](https://mlsanity.com/paper/2608.30858/) (2026)

## Abstract (arXiv)

Autonomous robots must navigate reliably in unknown environments even under compromised exteroceptive perception, or perception failures. Such failures often occur when harsh environments lead to degraded sensing, or when the perception algorithm misinterprets the scene due to limited generalization. In this paper, we model perception failures as invisible obstacles and pits, and train a reinforcement learning (RL) based local navigation policy to guide our legged robot. Unlike previous works relying on heuristics and anomaly detection to update navigational information, we train our navigation policy to reconstruct the environment information in the latent space from corrupted perception and react to perception failures end-to-end. To this end, we incorporate both proprioception and exteroception into our policy inputs, thereby enabling the policy to sense collisions on different body parts and pits, prompting corresponding reactions. We validate our approach in simulation and on the real quadruped robot ANYmal running in real-time (<10 ms CPU inference). In a quantitative comparison with existing heuristic-based locally reactive planners, our policy increases the success rate over 30% when facing perception failures. Project Page: https://bit.ly/45NBTuh.