# Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive Learning

arXiv: https://arxiv.org/abs/2103.03651 · HTML: https://mlsanity.com/paper/2103.03651/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [Self-Driving Cars: A Survey](https://mlsanity.com/paper/1901.04407/) (2019)
- [Contrastive Learning for Label-Efficient Semantic Segmentation](https://mlsanity.com/paper/2012.06985/) (2020)
- [Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning](https://mlsanity.com/paper/1912.03227/) (2019)
- [Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challenges](https://mlsanity.com/paper/1707.02432/) (2017)
- [Off-Road Drivable Area Extraction Using 3D LiDAR Data](https://mlsanity.com/paper/2003.04780/) (2020)

## Built on by (most cited first, 9 of 9)

- [RCA: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning](https://mlsanity.com/paper/2207.14460/) (2022)
- [Advancing Off-Road Autonomous Driving: The Large-Scale ORAD-3D Dataset and Comprehensive Benchmarks](https://mlsanity.com/paper/2510.16500/) (2025)
- [CoverNav: Cover Following Navigation Planning in Unstructured Outdoor Environment with Deep Reinforcement Learning](https://mlsanity.com/paper/2308.06594/) (2023)
- [Autonomous Driving in Unstructured Environments: How Far Have We Come?](https://mlsanity.com/paper/2410.07701/) (2024)
- [Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception](https://mlsanity.com/paper/2306.14601/) (2023)
- [TE-NeXt: A LiDAR-Based 3D Sparse Convolutional Network for Traversability Estimation](https://mlsanity.com/paper/2406.01395/) (2024)
- [METAVerse: Meta-Learning Traversability Cost Map for Off-Road Navigation](https://mlsanity.com/paper/2307.13991/) (2023)
- [FASTC: A Fast Attentional Framework for Semantic Traversability Classification Using Point Cloud](https://mlsanity.com/paper/2406.16564/) (2024)
- [A Cross-Scale Decoder with Token Refinement for Off-Road Semantic Segmentation](https://mlsanity.com/paper/2603.27931/) (2026)

## Abstract (arXiv)

Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot to traverse off-road is still an open question. This research proposes a contrastive learning based method. With a set of human-annotated anchor patches, a feature representation is learned to discriminate regions with different traversability, a method of fine-grained semantic segmentation and mapping is subsequently developed for off-road scene understanding. Experiments are conducted on a dataset of three driving segments that represent very diverse off-road scenes. An anchor accuracy of 89.8% is achieved by evaluating the matching with human-annotated image patches in cross-scene validation. Examined by associated 3D LiDAR data, the fine-grained segments of visual images are demonstrated to have different levels of toughness and terrain elevation, which represents their semantical meaningfulness. The resultant maps contain both fine-grained labels and confidence values, providing rich information to support a robot traversing complex off-road scenes.