# SoftSeg: Advantages of soft versus binary training for image segmentation

arXiv: https://arxiv.org/abs/2011.09041 · HTML: https://mlsanity.com/paper/2011.09041/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 26 of 26)

- [End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reduction](https://mlsanity.com/paper/2101.03244/) (2021)
- [ProtoGate: Prototype-based Neural Networks with Global-to-local Feature Selection for Tabular Biomedical Data](https://mlsanity.com/paper/2306.12330/) (2023)
- [BATFormer: Towards Boundary-Aware Lightweight Transformer for Efficient Medical Image Segmentation](https://mlsanity.com/paper/2206.14409/) (2022)
- [Label fusion and training methods for reliable representation of inter-rater uncertainty](https://mlsanity.com/paper/2202.07550/) (2022)
- [On the Optimal Combination of Cross-Entropy and Soft Dice Losses for Lesion Segmentation with Out-of-Distribution Robustness](https://mlsanity.com/paper/2209.06078/) (2022)
- [Impact of individual rater style on deep learning uncertainty in medical imaging segmentation](https://mlsanity.com/paper/2105.02197/) (2021)
- [Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning to Segment Driveability in Egocentric Images](https://mlsanity.com/paper/2109.07245/) (2021)
- [On Image Segmentation With Noisy Labels: Characterization and Volume Properties of the Optimal Solutions to Accuracy and Dice](https://mlsanity.com/paper/2206.06484/) (2022)
- [Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels](https://mlsanity.com/paper/2303.16296/) (2023)
- [SemSegLoss: A python package of loss functions for semantic segmentation](https://mlsanity.com/paper/2106.05844/) (2021)
- [Segmentation of Multiple Sclerosis Lesions across Hospitals: Learn Continually or Train from Scratch?](https://mlsanity.com/paper/2210.15091/) (2022)
- [Weakly Supervised Medical Image Segmentation With Soft Labels and Noise Robust Loss](https://mlsanity.com/paper/2209.08172/) (2022)
- [Towards contrast-agnostic soft segmentation of the spinal cord](https://mlsanity.com/paper/2310.15402/) (2023)
- [BackSplit: The Importance of Sub-dividing the Background in Biomedical Lesion Segmentation](https://mlsanity.com/paper/2511.19394/) (2025)
- [Team NeuroPoly: Description of the Pipelines for the MICCAI 2021 MS New Lesions Segmentation Challenge](https://mlsanity.com/paper/2109.05409/) (2021)
- [From CAD models to soft point cloud labels: An automatic annotation pipeline for cheaply supervised 3D semantic segmentation](https://mlsanity.com/paper/2302.03114/) (2023)
- [Noisy Image Segmentation With Soft-Dice](https://mlsanity.com/paper/2304.00801/) (2023)
- [Multi-Domain Data Aggregation for Axon and Myelin Segmentation in Histology Images](https://mlsanity.com/paper/2409.11552/) (2024)
- [RAPID-Net: Accurate Pocket Identification for Binding-Site-Agnostic Docking](https://mlsanity.com/paper/2502.02371/) (2025)
- [Ensemble CNN and Uncertainty Modeling to Improve Automatic Identification/Segmentation of Multiple Sclerosis Lesions in Magnetic Resonance Imaging](https://mlsanity.com/paper/2108.11791/) (2021)
- [Marginal Thresholding in Noisy Image Segmentation](https://mlsanity.com/paper/2304.04116/) (2023)
- [Towards Efficient and Accurate CT Segmentation via Edge-Preserving Probabilistic Downsampling](https://mlsanity.com/paper/2404.03991/) (2024)
- [Confidence-Based Annotation Of Brain Tumours In Ultrasound](https://mlsanity.com/paper/2502.15484/) (2025)
- [Label Uncertainty for Ultrasound Segmentation](https://mlsanity.com/paper/2508.15635/) (2025)
- [Optimization in Sparse 2D to Dense 3D Weakly Supervised Learning: Application to Multi-Label Segmentation of Large ex vivo MRI Data](https://mlsanity.com/paper/2605.12753/) (2026)
- [An Ultra-Widefield Swept-Source OCTA Dataset and a Polar-Gated Mamba Network for Retinal Vessel Segmentation](https://mlsanity.com/paper/2609.12574/) (2026)

## Abstract (arXiv)

Most image segmentation algorithms are trained on binary masks formulated as a classification task per pixel. However, in applications such as medical imaging, this "black-and-white" approach is too constraining because the contrast between two tissues is often ill-defined, i.e., the voxels located on objects' edges contain a mixture of tissues. Consequently, assigning a single "hard" label can result in a detrimental approximation. Instead, a soft prediction containing non-binary values would overcome that limitation. We introduce SoftSeg, a deep learning training approach that takes advantage of soft ground truth labels, and is not bound to binary predictions. SoftSeg aims at solving a regression instead of a classification problem. This is achieved by using (i) no binarization after preprocessing and data augmentation, (ii) a normalized ReLU final activation layer (instead of sigmoid), and (iii) a regression loss function (instead of the traditional Dice loss). We assess the impact of these three features on three open-source MRI segmentation datasets from the spinal cord gray matter, the multiple sclerosis brain lesion, and the multimodal brain tumor segmentation challenges. Across multiple cross-validation iterations, SoftSeg outperformed the conventional approach, leading to an increase in Dice score of 2.0% on the gray matter dataset (p=0.001), 3.3% for the MS lesions, and 6.5% for the brain tumors. SoftSeg produces consistent soft predictions at tissues' interfaces and shows an increased sensitivity for small objects. The richness of soft labels could represent the inter-expert variability, the partial volume effect, and complement the model uncertainty estimation. The developed training pipeline can easily be incorporated into most of the existing deep learning architectures. It is already implemented in the freely-available deep learning toolbox ivadomed (https://ivadomed.org).