# RankSEG: A Consistent Ranking-based Framework for Segmentation

arXiv: https://arxiv.org/abs/2206.13086 · HTML: https://mlsanity.com/paper/2206.13086/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Tversky loss function for image segmentation using 3D fully convolutional deep networks](https://mlsanity.com/paper/1706.05721/) (2017)
- [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017)
- [DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation](https://mlsanity.com/paper/1707.00652/) (2017)
- [On Focal Loss for Class-Posterior Probability Estimation: A Theoretical Perspective](https://mlsanity.com/paper/2011.09172/) (2020)
- [Deep Probability Estimation](https://mlsanity.com/paper/2111.10734/) (2021)
- [On the relationship between calibrated predictors and unbiased volume estimation](https://mlsanity.com/paper/2112.12560/) (2021)
- [Calibrated Surrogate Maximization of Linear-fractional Utility in Binary Classification](https://mlsanity.com/paper/1905.12511/) (2019)
- [Optimization with soft Dice can lead to a volumetric bias](https://mlsanity.com/paper/1911.02278/) (2019)
- [SemSegLoss: A python package of loss functions for semantic segmentation](https://mlsanity.com/paper/2106.05844/) (2021)

## Built on by (most cited first, 7 of 7)

- [Principled Algorithms for Optimizing Generalized Metrics in Binary Classification](https://mlsanity.com/paper/2512.23133/) (2025)
- [Soft Dice Confidence: A Near-Optimal Confidence Estimator for Selective Prediction in Semantic Segmentation](https://mlsanity.com/paper/2402.10665/) (2024)
- [EnsLoss: Stochastic Calibrated Loss Ensembles for Preventing Overfitting in Classification](https://mlsanity.com/paper/2409.00908/) (2024)
- [RankSEG-RMA: An Efficient Segmentation Algorithm via Reciprocal Moment Approximation](https://mlsanity.com/paper/2510.15362/) (2025)
- [CRISP: Constrained Refinement via Iterative Squeezing Process for Robust Medical Image Segmentation under Domain Shift](https://mlsanity.com/paper/2607.15231/) (2026)
- [Exponential Convex Calibration Dimension for the Multi-Label Jaccard Measure](https://mlsanity.com/paper/2608.13549/) (2026)
- [On the Relaxation of Conditional Independence Assumption for Image Segmentation](https://mlsanity.com/paper/2609.38930/) (2026)

## Abstract (arXiv)

Segmentation has emerged as a fundamental field of computer vision and natural language processing, which assigns a label to every pixel/feature to extract regions of interest from an image/text. To evaluate the performance of segmentation, the Dice and IoU metrics are used to measure the degree of overlap between the ground truth and the predicted segmentation. In this paper, we establish a theoretical foundation of segmentation with respect to the Dice/IoU metrics, including the Bayes rule and Dice-/IoU-calibration, analogous to classification-calibration or Fisher consistency in classification. We prove that the existing thresholding-based framework with most operating losses are not consistent with respect to the Dice/IoU metrics, and thus may lead to a suboptimal solution. To address this pitfall, we propose a novel consistent ranking-based framework, namely RankDice/RankIoU, inspired by plug-in rules of the Bayes segmentation rule. Three numerical algorithms with GPU parallel execution are developed to implement the proposed framework in large-scale and high-dimensional segmentation. We study statistical properties of the proposed framework. We show it is Dice-/IoU-calibrated, and its excess risk bounds and the rate of convergence are also provided. The numerical effectiveness of RankDice/mRankDice is demonstrated in various simulated examples and Fine-annotated CityScapes, Pascal VOC and Kvasir-SEG datasets with state-of-the-art deep learning architectures.