# Scene-Agnostic Traversability Labeling and Estimation via a Multimodal Self-supervised Framework

arXiv: https://arxiv.org/abs/2508.18249 · HTML: https://mlsanity.com/paper/2508.18249/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 26 of 26)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2106.15277/) (2021)
- [GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2103.04233/) (2021)
- [Fast Traversability Estimation for Wild Visual Navigation](https://mlsanity.com/paper/2305.08510/) (2023)
- [Elevation Mapping for Locomotion and Navigation using GPU](https://mlsanity.com/paper/2204.12876/) (2022)
- [Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning](https://mlsanity.com/paper/1912.03227/) (2019)
- [V-STRONG: Visual Self-Supervised Traversability Learning for Off-road Navigation](https://mlsanity.com/paper/2312.16016/) (2023)
- [Learning Off-Road Terrain Traversability with Self-Supervisions Only](https://mlsanity.com/paper/2305.18896/) (2023)
- [GrASPE: Graph based Multimodal Fusion for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2209.05722/) (2022)
- [Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision](https://mlsanity.com/paper/2404.07110/) (2024)
- [SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Clouds](https://mlsanity.com/paper/2407.11569/) (2024)
- [Learning-on-the-Drive: Self-supervised Adaptation of Visual Offroad Traversability Models](https://mlsanity.com/paper/2306.15226/) (2023)
- [Traversability Analysis for Autonomous Driving in Complex Environment: A LiDAR-based Terrain Modeling Approach](https://mlsanity.com/paper/2307.02060/) (2023)
- [SNE-RoadSegV2: Advancing Heterogeneous Feature Fusion and Fallibility Awareness for Freespace Detection](https://mlsanity.com/paper/2402.18918/) (2024)
- [PUTN: A Plane-fitting based Uneven Terrain Navigation Framework](https://mlsanity.com/paper/2203.04541/) (2022)
- [Contrastive Label Disambiguation for Self-Supervised Terrain Traversability Learning in Off-Road Environments](https://mlsanity.com/paper/2307.02871/) (2023)
- [Follow the Footprints: Self-supervised Traversability Estimation for Off-road Vehicle Navigation based on Geometric and Visual Cues](https://mlsanity.com/paper/2402.15363/) (2024)
- [Learning Semantic Traversability with Egocentric Video and Automated Annotation Strategy](https://mlsanity.com/paper/2406.02989/) (2024)
- [IMOST: Incremental Memory Mechanism with Online Self-Supervision for Continual Traversability Learning](https://mlsanity.com/paper/2409.14070/) (2024)
- [SALT: A Flexible Semi-Automatic Labeling Tool for General LiDAR Point Clouds with Cross-Scene Adaptability and 4D Consistency](https://mlsanity.com/paper/2503.23980/) (2025)
- [Gaussian Process-based Traversability Analysis for Terrain Mapless Navigation](https://mlsanity.com/paper/2403.19010/) (2024)
- [Trajectory-based Road Autolabeling with Lidar-Camera Fusion in Winter Conditions](https://mlsanity.com/paper/2412.02370/) (2024)
- [Self-supervised cost of transport estimation for multimodal path planning](https://mlsanity.com/paper/2412.06101/) (2024)

## Abstract (arXiv)

Traversability estimation is critical for enabling robots to navigate across diverse terrains and environments. While recent self-supervised learning methods achieve promising results, they often fail to capture the characteristics of non-traversable regions. Moreover, most prior works concentrate on a single modality, overlooking the complementary strengths offered by integrating heterogeneous sensory modalities for more robust traversability estimation. To address these limitations, we propose a multimodal self-supervised framework for traversability labeling and estimation. First, our annotation pipeline integrates footprint, LiDAR, and camera data as prompts for a vision foundation model, generating traversability labels that account for both semantic and geometric cues. Then, leveraging these labels, we train a dual-stream network that jointly learns from different modalities in a decoupled manner, enhancing its capacity to recognize diverse traversability patterns. In addition, we incorporate sparse LiDAR-based supervision to mitigate the noise introduced by pseudo labels. Finally, extensive experiments conducted across urban, off-road, and campus environments demonstrate the effectiveness of our approach. The proposed automatic labeling method consistently achieves around 88% IoU across diverse datasets. Compared to existing self-supervised state-of-the-art methods, our multimodal traversability estimation network yields consistently higher IoU, improving by 1.6-3.5% on all evaluated datasets.