# Soft labelling for semantic segmentation: Bringing coherence to label down-sampling

arXiv: https://arxiv.org/abs/2302.13961 · HTML: https://mlsanity.com/paper/2302.13961/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 30 of 31)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Simultaneous Deep Transfer Across Domains and Tasks](https://mlsanity.com/paper/1510.02192/) (2015)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation](https://mlsanity.com/paper/2007.09222/) (2020)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1904.02216/) (2019)
- [ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network](https://mlsanity.com/paper/1811.11431/) (2018)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [Deep Hierarchical Semantic Segmentation](https://mlsanity.com/paper/2203.14335/) (2022)
- [In-Place Activated BatchNorm for Memory-Optimized Training of DNNs](https://mlsanity.com/paper/1712.02616/) (2017)
- [InverseForm: A Loss Function for Structured Boundary-Aware Segmentation](https://mlsanity.com/paper/2104.02745/) (2021)
- [LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1905.02423/) (2019)
- [Region Mutual Information Loss for Semantic Segmentation](https://mlsanity.com/paper/1910.12037/) (2019)
- [Progressive Semantic Segmentation](https://mlsanity.com/paper/2104.03778/) (2021)
- [Efficient Segmentation: Learning Downsampling Near Semantic Boundaries](https://mlsanity.com/paper/1907.07156/) (2019)
- [Graph-guided Architecture Search for Real-time Semantic Segmentation](https://mlsanity.com/paper/1909.06793/) (2019)
- [Transformer Scale Gate for Semantic Segmentation](https://mlsanity.com/paper/2205.07056/) (2022)
- [Dynamic Focus-aware Positional Queries for Semantic Segmentation](https://mlsanity.com/paper/2204.01244/) (2022)
- [MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation](https://mlsanity.com/paper/2103.13044/) (2021)
- [Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation](https://mlsanity.com/paper/1905.08720/) (2019)

## Built on by (most cited first, 5 of 5)

- [Gradient-based Class Weighting for Unsupervised Domain Adaptation in Dense Prediction Visual Tasks](https://mlsanity.com/paper/2407.01327/) (2024)
- [Layer-wise Model Merging for Unsupervised Domain Adaptation in Segmentation Tasks](https://mlsanity.com/paper/2409.15813/) (2024)
- [Leveraging Contrastive Learning for Semantic Segmentation with Consistent Labels Across Varying Appearances](https://mlsanity.com/paper/2412.16592/) (2024)
- [Towards Efficient and Accurate CT Segmentation via Edge-Preserving Probabilistic Downsampling](https://mlsanity.com/paper/2404.03991/) (2024)
- [Image-level Regression for Uncertainty-aware Retinal Image Segmentation](https://mlsanity.com/paper/2405.16815/) (2024)

## Abstract (arXiv)

In semantic segmentation, training data down-sampling is commonly performed due to limited resources, the need to adapt image size to the model input, or improve data augmentation. This down-sampling typically employs different strategies for the image data and the annotated labels. Such discrepancy leads to mismatches between the down-sampled color and label images. Hence, the training performance significantly decreases as the down-sampling factor increases. In this paper, we bring together the down-sampling strategies for the image data and the training labels. To that aim, we propose a novel framework for label down-sampling via soft-labeling that better conserves label information after down-sampling. Therefore, fully aligning soft-labels with image data to keep the distribution of the sampled pixels. This proposal also produces reliable annotations for under-represented semantic classes. Altogether, it allows training competitive models at lower resolutions. Experiments show that the proposal outperforms other down-sampling strategies. Moreover, state-of-the-art performance is achieved for reference benchmarks, but employing significantly less computational resources than foremost approaches. This proposal enables competitive research for semantic segmentation under resource constraints.