# Semi-Supervised Active Learning for Semantic Segmentation in Unknown Environments Using Informative Path Planning

arXiv: https://arxiv.org/abs/2312.04402 · HTML: https://mlsanity.com/paper/2312.04402/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 24 of 24)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Semi-Supervised Semantic Segmentation with Cross-Consistency Training](https://mlsanity.com/paper/2003.09005/) (2020)
- [Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning](https://mlsanity.com/paper/1803.04765/) (2018)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation](https://mlsanity.com/paper/2107.11279/) (2021)
- [Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.12940/) (2021)
- [Learning to Map for Active Semantic Goal Navigation](https://mlsanity.com/paper/2106.15648/) (2021)
- [SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency](https://mlsanity.com/paper/2112.01001/) (2021)
- [All you need are a few pixels: semantic segmentation with PixelPick](https://mlsanity.com/paper/2104.06394/) (2021)
- [Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs](https://mlsanity.com/paper/2007.12640/) (2020)
- [An informative path planning framework for UAV-based terrain monitoring](https://mlsanity.com/paper/1809.03870/) (2018)
- [From colouring-in to pointillism: revisiting semantic segmentation supervision](https://mlsanity.com/paper/2210.14142/) (2022)
- [DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing](https://mlsanity.com/paper/2201.01047/) (2022)
- [Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing](https://mlsanity.com/paper/2109.13570/) (2021)
- [Embodied Active Domain Adaptation for Semantic Segmentation via Informative Path Planning](https://mlsanity.com/paper/2203.00549/) (2022)
- [Gaussian Process Autonomous Mapping and Exploration for Range Sensing Mobile Robots](https://mlsanity.com/paper/1605.00335/) (2016)
- [An Informative Path Planning Framework for Active Learning in UAV-based Semantic Mapping](https://mlsanity.com/paper/2302.03347/) (2023)
- [Continual Adaptation of Semantic Segmentation using Complementary 2D-3D Data Representations](https://mlsanity.com/paper/2111.02156/) (2021)
- [Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensing](https://mlsanity.com/paper/2209.07660/) (2022)
- [Active Learning for UAV-based Semantic Mapping](https://mlsanity.com/paper/1908.11157/) (2019)
- [Informative Path Planning for Active Learning in Aerial Semantic Mapping](https://mlsanity.com/paper/2203.01652/) (2022)

## Built on by (most cited first, 6 of 6)

- [Enhancing Large Vision Model in Street Scene Semantic Understanding through Leveraging Posterior Optimization Trajectory](https://mlsanity.com/paper/2501.01710/) (2025)
- [FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving](https://mlsanity.com/paper/2505.00318/) (2025)
- [Perception Without Vision for Trajectory Prediction: Ego Vehicle Dynamics as Scene Representation for Efficient Active Learning in Autonomous Driving](https://mlsanity.com/paper/2405.09049/) (2024)
- [A2LC: Active and Automated Label Correction for Semantic Segmentation](https://mlsanity.com/paper/2506.11599/) (2025)
- [Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator](https://mlsanity.com/paper/2502.02972/) (2025)
- [FedDSR: Federated Deep Supervision and Regularization Towards Autonomous Driving](https://mlsanity.com/paper/2512.06676/) (2025)

## Abstract (arXiv)

Semantic segmentation enables robots to perceive and reason about their environments beyond geometry. Most of such systems build upon deep learning approaches. As autonomous robots are commonly deployed in initially unknown environments, pre-training on static datasets cannot always capture the variety of domains and limits the robot's perception performance during missions. Recently, self-supervised and fully supervised active learning methods emerged to improve a robot's vision. These approaches rely on large in-domain pre-training datasets or require substantial human labelling effort. We propose a planning method for semi-supervised active learning of semantic segmentation that substantially reduces human labelling requirements compared to fully supervised approaches. We leverage an adaptive map-based planner guided towards the frontiers of unexplored space with high model uncertainty collecting training data for human labelling. A key aspect of our approach is to combine the sparse high-quality human labels with pseudo labels automatically extracted from highly certain environment map areas. Experimental results show that our method reaches segmentation performance close to fully supervised approaches with drastically reduced human labelling effort while outperforming self-supervised approaches.