# Watch Your STEPP: Semantic Traversability Estimation using Pose Projected Features

arXiv: https://arxiv.org/abs/2501.17594 · HTML: https://mlsanity.com/paper/2501.17594/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 29 Jan 2025.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [ViPlanner: Visual Semantic Imperative Learning for Local Navigation](https://mlsanity.com/paper/2310.00982/) (2023)
- [V-STRONG: Visual Self-Supervised Traversability Learning for Off-road Navigation](https://mlsanity.com/paper/2312.16016/) (2023)
- [One-Shot Transfer of Affordance Regions? AffCorrs!](https://mlsanity.com/paper/2209.07147/) (2022)
- [TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation](https://mlsanity.com/paper/2303.15771/) (2023)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [Autonomous Exploration Development Environment and the Planning Algorithms](https://mlsanity.com/paper/2110.14573/) (2021)
- [RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving](https://mlsanity.com/paper/2402.19341/) (2024)
- [ScaTE: A Scalable Framework for Self-Supervised Traversability Estimation in Unstructured Environments](https://mlsanity.com/paper/2209.06522/) (2022)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [DiPPeR: Diffusion-based 2D Path Planner applied on Legged Robots](https://mlsanity.com/paper/2310.07842/) (2023)
- [Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision](https://mlsanity.com/paper/2404.07110/) (2024)
- [These Maps Are Made For Walking: Real-Time Terrain Property Estimation for Mobile Robots](https://mlsanity.com/paper/2205.12925/) (2022)
- [DiPPeST: Diffusion-based Path Planner for Synthesizing Trajectories Applied on Quadruped Robots](https://mlsanity.com/paper/2405.19232/) (2024)
- [PIETRA: Physics-Informed Evidential Learning for Traversing Out-of-Distribution Terrain](https://mlsanity.com/paper/2409.03005/) (2024)
- [Learning Semantic Traversability with Egocentric Video and Automated Annotation Strategy](https://mlsanity.com/paper/2406.02989/) (2024)
- [LiteVLoc: Map-Lite Visual Localization for Image Goal Navigation](https://mlsanity.com/paper/2410.04419/) (2024)
- [IN-Sight: Interactive Navigation through Sight](https://mlsanity.com/paper/2408.00343/) (2024)

## Built on by (most cited first, 4 of 4)

- [OneOcc: Semantic Occupancy Prediction for Legged Robots with a Single Panoramic Camera](https://mlsanity.com/paper/2511.03571/) (2025)
- [From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments](https://mlsanity.com/paper/2605.29565/) (2026)
- [SyNeT: Synthetic Negatives for Traversability Learning](https://mlsanity.com/paper/2602.00814/) (2026)
- [Which Terrain Is Better? Preference Learning with VLM Prototypes for Off-Road Traversability Ranking](https://mlsanity.com/paper/2609.23673/) (2026)

## Abstract (arXiv)

Understanding the traversability of terrain is essential for autonomous robot navigation, particularly in unstructured environments such as natural landscapes. Although traditional methods, such as occupancy mapping, provide a basic framework, they often fail to account for the complex mobility capabilities of some platforms such as legged robots. In this work, we propose a method for estimating terrain traversability by learning from demonstrations of human walking. Our approach leverages dense, pixel-wise feature embeddings generated using the DINOv2 vision Transformer model, which are processed through an encoder-decoder MLP architecture to analyze terrain segments. The averaged feature vectors, extracted from the masked regions of interest, are used to train the model in a reconstruction-based framework. By minimizing reconstruction loss, the network distinguishes between familiar terrain with a low reconstruction error and unfamiliar or hazardous terrain with a higher reconstruction error. This approach facilitates the detection of anomalies, allowing a legged robot to navigate more effectively through challenging terrain. We run real-world experiments on the ANYmal legged robot both indoor and outdoor to prove our proposed method. The code is open-source, while video demonstrations can be found on our website: https://rpl-cs-ucl.github.io/STEPP