An Active and Contrastive Learning Framework for Fine-Grained Off-Road Semantic Segmentation
Not on the map. Why?
Abstract (arXiv)
Off-road semantic segmentation with fine-grained labels is necessary for autonomous vehicles to understand driving scenes, as the coarse-grained road detection can not satisfy off-road vehicles with various mechanical properties. Fine-grained semantic segmentation in off-road scenes usually has no unified category definition due to ambiguous nature environments, and the cost of pixel-wise labeling is extremely high. Furthermore, semantic properties of off-road scenes can be very changeable due to various precipitations, temperature, defoliation, etc. To address these challenges, this research proposes an active and contrastive learning-based method that does not rely on pixel-wise labels, but only on patch-based weak annotations for model learning. There is no need for predefined semantic categories, the contrastive learning-based feature representation and adaptive clustering will discover the category model from scene data. In order to actively adapt to new scenes, a risk evaluation method is proposed to discover and select hard frames with high-risk predictions for supplemental labeling, so as to update the model efficiently. Experiments conducted on our self-developed off-road dataset and DeepScene dataset demonstrate that fine-grained semantic segmentation can be learned with only dozens of weakly labeled frames, and the model can efficiently adapt across scenes by weak supervision, while achieving almost the same level of performance as typical fully supervised baselines.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 43 of its 91 references that are in the map. First posted on arXiv 18 Feb 2022.
Builds on
43 of 91 references.
References · 43
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2020Supervised Contrastive Learningcited by 2,803
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
- 2019Contrastive Multiview Codingcited by 1,297
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2020Image Segmentation Using Deep Learning: A Surveycited by 659
- 2020A Survey of Deep Active Learningcited by 547
31 more
- 2019Variational Adversarial Active Learningcited by 284
- 2021Exploring Cross-Image Pixel Contrast for Semantic Segmentationcited by 242
- 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
- 2015ParseNet: Looking Wider to See Bettercited by 176
- 2017Cost-Effective Active Learning for Deep Image Classificationcited by 165
- 2019Self-Driving Cars: A Surveycited by 147
- 2017Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentationcited by 145
- 2021Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bankcited by 125
- 2021Semi-supervised Semantic Segmentation with Directional Context-aware Consistencycited by 112
- 2019ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentationcited by 95
- 2020Contrastive Learning for Label-Efficient Semantic Segmentationcited by 93
- 2018Self-Supervised Model Adaptation for Multimodal Semantic Segmentationcited by 73
- 2020Reinforced active learning for image segmentationcited by 56
- 2021Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentationcited by 56
- 2021GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environmentscited by 51
- 2018CEREALS - Cost-Effective REgion-based Active Learning for Semantic Segmentationcited by 47
- 2017Cost-Effective Active Learning for Melanoma Segmentationcited by 40
- 2019Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learningcited by 37
- 2020DEAL: Difficulty-aware Active Learning for Semantic Segmentationcited by 32
- 2021Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learningcited by 30
- 2017Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challengescited by 24
- 2020MetaBox+: A new Region Based Active Learning Method for Semantic Segmentation using Priority Mapscited by 23
- 2020Embodied Visual Active Learning for Semantic Segmentationcited by 19
- 2021Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentationcited by 19
- 2021OFFSEG: A Semantic Segmentation Framework For Off-Road Drivingcited by 18
- 2021Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentationcited by 14
- 2021Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentationcited by 14
- 2020Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Studycited by 12
- 2021Memory-based Semantic Segmentation for Off-road Unstructured Natural Environmentscited by 6
- 2020Off-Road Drivable Area Extraction Using 3D LiDAR Datacited by 4
- 2020Evaluation of 3D CNN Semantic Mapping for Rover Navigationcited by 1
Built on by
Citing papers per year: 2024: 2.
2 papers.