# Importance of Self-Consistency in Active Learning for Semantic Segmentation

arXiv: https://arxiv.org/abs/2008.01860 · HTML: https://mlsanity.com/paper/2008.01860/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 21 of 21)

- [All you need are a few pixels: semantic segmentation with PixelPick](https://mlsanity.com/paper/2104.06394/) (2021)
- [Hyperbolic Active Learning for Semantic Segmentation under Domain Shift](https://mlsanity.com/paper/2306.11180/) (2023)
- [Semantic Segmentation with Active Semi-Supervised Learning](https://mlsanity.com/paper/2203.10730/) (2022)
- [Class-Balanced Active Learning for Image Classification](https://mlsanity.com/paper/2110.04543/) (2021)
- [Adaptive Superpixel for Active Learning in Semantic Segmentation](https://mlsanity.com/paper/2303.16817/) (2023)
- [Best Practices in Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2302.04075/) (2023)
- [Multi-Task Consistency for Active Learning](https://mlsanity.com/paper/2306.12398/) (2023)
- [Active Learning for Semantic Segmentation with Multi-class Label Query](https://mlsanity.com/paper/2309.09319/) (2023)
- [Deep Active Learning for Computer Vision: Past and Future](https://mlsanity.com/paper/2211.14819/) (2022)
- [MEAL: Manifold Embedding-based Active Learning](https://mlsanity.com/paper/2106.11858/) (2021)
- [Correlation-aware active learning for surgery video segmentation](https://mlsanity.com/paper/2311.08811/) (2023)
- [Self-supervised Learning for Panoptic Segmentation of Multiple Fruit Flower Species](https://mlsanity.com/paper/2209.04618/) (2022)
- [Semantic Segmentation with Active Semi-Supervised Representation Learning](https://mlsanity.com/paper/2210.08403/) (2022)
- [LiDAR dataset distillation within bayesian active learning framework: Understanding the effect of data augmentation](https://mlsanity.com/paper/2202.02661/) (2022)
- [Active Learning with Context Sampling and One-vs-Rest Entropy for Semantic Segmentation](https://mlsanity.com/paper/2412.06470/) (2024)
- [Tracking Passengers and Baggage Items using Multiple Overhead Cameras at Security Checkpoints](https://mlsanity.com/paper/2301.00190/) (2022)
- [Embodied Learning for Lifelong Visual Perception](https://mlsanity.com/paper/2112.14084/) (2021)
- [Evaluating the effect of data augmentation and BALD heuristics on distillation of Semantic-KITTI dataset](https://mlsanity.com/paper/2302.10679/) (2023)
- [Maturity-Aware Active Learning for Semantic Segmentation with Hierarchically-Adaptive Sample Assessment](https://mlsanity.com/paper/2308.14904/) (2023)
- [Key Patch Proposer: Key Patches Contain Rich Information](https://mlsanity.com/paper/2402.11458/) (2024)
- [A Self Consistency Based Reranking for Narrative Question Answering](https://mlsanity.com/paper/2606.15741/) (2026)

## Abstract (arXiv)

We address the task of active learning in the context of semantic segmentation and show that self-consistency can be a powerful source of self-supervision to greatly improve the performance of a data-driven model with access to only a small amount of labeled data. Self-consistency uses the simple observation that the results of semantic segmentation for a specific image should not change under transformations like horizontal flipping (i.e., the results should only be flipped). In other words, the output of a model should be consistent under equivariant transformations. The self-supervisory signal of self-consistency is particularly helpful during active learning since the model is prone to overfitting when there is only a small amount of labeled training data. In our proposed active learning framework, we iteratively extract small image patches that need to be labeled, by selecting image patches that have high uncertainty (high entropy) under equivariant transformations. We enforce pixel-wise self-consistency between the outputs of segmentation network for each image and its transformation (horizontally flipped) to utilize the rich self-supervisory information and reduce the uncertainty of the network. In this way, we are able to find the image patches over which the current model struggles the most to classify. By iteratively training over these difficult image patches, our experiments show that our active learning approach reaches ~96% of the top performance of a model trained on all data, by using only 12% of the total data on benchmark semantic segmentation datasets (e.g., CamVid and Cityscapes).