# Self-Supervised Traversability Prediction by Learning to Reconstruct Safe Terrain

arXiv: https://arxiv.org/abs/2208.01329 · HTML: https://mlsanity.com/paper/2208.01329/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 2 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [Self-Supervised Model Adaptation for Multimodal Semantic Segmentation](https://mlsanity.com/paper/1808.03833/) (2018)
- [Misbehaviour Prediction for Autonomous Driving Systems](https://mlsanity.com/paper/1910.04443/) (2019)
- [Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning](https://mlsanity.com/paper/1912.03227/) (2019)
- [Safe Robot Navigation via Multi-Modal Anomaly Detection](https://mlsanity.com/paper/2001.07934/) (2020)
- [Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy](https://mlsanity.com/paper/1610.01238/) (2016)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [G-VOM: A GPU Accelerated Voxel Off-Road Mapping System](https://mlsanity.com/paper/2109.13176/) (2021)
- [Learning Long-Range Perception Using Self-Supervision from Short-Range Sensors and Odometry](https://mlsanity.com/paper/1809.07207/) (2018)

## Built on by (most cited first, 22 of 22)

- [Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures](https://mlsanity.com/paper/2505.00779/) (2025)
- [Learning Off-Road Terrain Traversability with Self-Supervisions Only](https://mlsanity.com/paper/2305.18896/) (2023)
- [RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving](https://mlsanity.com/paper/2402.19341/) (2024)
- [ScaTE: A Scalable Framework for Self-Supervised Traversability Estimation in Unstructured Environments](https://mlsanity.com/paper/2209.06522/) (2022)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision](https://mlsanity.com/paper/2404.07110/) (2024)
- [PIETRA: Physics-Informed Evidential Learning for Traversing Out-of-Distribution Terrain](https://mlsanity.com/paper/2409.03005/) (2024)
- [Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles](https://mlsanity.com/paper/2307.09027/) (2023)
- [Learning Semantic Traversability with Egocentric Video and Automated Annotation Strategy](https://mlsanity.com/paper/2406.02989/) (2024)
- [RoadRunner M&M -- Learning Multi-range Multi-resolution Traversability Maps for Autonomous Off-road Navigation](https://mlsanity.com/paper/2409.10940/) (2024)
- [ROD: RGB-Only Fast and Efficient Off-road Freespace Detection](https://mlsanity.com/paper/2508.08697/) (2025)
- [Few-shot Semantic Learning for Robust Multi-Biome 3D Semantic Mapping in Off-Road Environments](https://mlsanity.com/paper/2411.06632/) (2024)
- [WildOS: Open-Vocabulary Object Search in the Wild](https://mlsanity.com/paper/2602.19308/) (2026)
- [Trajectory-based Road Autolabeling with Lidar-Camera Fusion in Winter Conditions](https://mlsanity.com/paper/2412.02370/) (2024)
- [Motion planning for off-road autonomous driving based on human-like cognition and weight adaptation](https://mlsanity.com/paper/2404.17820/) (2024)
- [Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling](https://mlsanity.com/paper/2405.01114/) (2024)
- [Self-supervised cost of transport estimation for multimodal path planning](https://mlsanity.com/paper/2412.06101/) (2024)
- [TADAP: Trajectory-Aided Drivable area Auto-labeling with Pre-trained self-supervised features in winter driving conditions](https://mlsanity.com/paper/2312.12954/) (2023)
- [Label Correction for Road Segmentation Using Road-side Cameras](https://mlsanity.com/paper/2502.01281/) (2025)
- [SyNeT: Synthetic Negatives for Traversability Learning](https://mlsanity.com/paper/2602.00814/) (2026)
- [Towards All-Day Perception for Off-Road Driving: A Large-Scale Multispectral Dataset and Comprehensive Benchmark](https://mlsanity.com/paper/2604.27499/) (2026)
- [Tracking the Invisible: Self-Supervised Wheel Track Detection with Thermal-RGB Fusion](https://mlsanity.com/paper/2610.13872/) (2026)

## Abstract (arXiv)

Navigating off-road with a fast autonomous vehicle depends on a robust perception system that differentiates traversable from non-traversable terrain. Typically, this depends on a semantic understanding which is based on supervised learning from images annotated by a human expert. This requires a significant investment in human time, assumes correct expert classification, and small details can lead to misclassification. To address these challenges, we propose a method for predicting high- and low-risk terrains from only past vehicle experience in a self-supervised fashion. First, we develop a tool that projects the vehicle trajectory into the front camera image. Second, occlusions in the 3D representation of the terrain are filtered out. Third, an autoencoder trained on masked vehicle trajectory regions identifies low- and high-risk terrains based on the reconstruction error. We evaluated our approach with two models and different bottleneck sizes with two different training and testing sites with a fourwheeled off-road vehicle. Comparison with two independent test sets of semantic labels from similar terrain as training sites demonstrates the ability to separate the ground as low-risk and the vegetation as high-risk with 81.1% and 85.1% accuracy.