# Uncertainty Estimation in Instance Segmentation with Star-convex Shapes

arXiv: https://arxiv.org/abs/2309.10513 · HTML: https://mlsanity.com/paper/2309.10513/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [Gland Segmentation in Colon Histology Images: The GlaS Challenge Contest](https://mlsanity.com/paper/1603.00275/) (2016)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- [Dropout Sampling for Robust Object Detection in Open-Set Conditions](https://mlsanity.com/paper/1710.06677/) (2017)
- [The Limits and Potentials of Deep Learning for Robotics](https://mlsanity.com/paper/1804.06557/) (2018)
- [Evaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detection](https://mlsanity.com/paper/1809.06006/) (2018)
- [Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector](https://mlsanity.com/paper/1901.10609/) (2019)
- [Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy](https://mlsanity.com/paper/1908.03636/) (2019)
- [Cost-Effective Active Learning for Melanoma Segmentation](https://mlsanity.com/paper/1711.09168/) (2017)
- [Straight to Shapes: Real-time Detection of Encoded Shapes](https://mlsanity.com/paper/1611.07932/) (2016)
- [Bubble identification from images with machine learning methods](https://mlsanity.com/paper/2202.03107/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Reliable AI-Based Histological Staining: A Systematic Study of Scaling and Uncertainty in Unpaired Generative Models](https://mlsanity.com/paper/2608.24626/) (2026)

## Abstract (arXiv)

Instance segmentation has witnessed promising advancements through deep neural network-based algorithms. However, these models often exhibit incorrect predictions with unwarranted confidence levels. Consequently, evaluating prediction uncertainty becomes critical for informed decision-making. Existing methods primarily focus on quantifying uncertainty in classification or regression tasks, lacking emphasis on instance segmentation. Our research addresses the challenge of estimating spatial certainty associated with the location of instances with star-convex shapes. Two distinct clustering approaches are evaluated which compute spatial and fractional certainty per instance employing samples by the Monte-Carlo Dropout or Deep Ensemble technique. Our study demonstrates that combining spatial and fractional certainty scores yields improved calibrated estimation over individual certainty scores. Notably, our experimental results show that the Deep Ensemble technique alongside our novel radial clustering approach proves to be an effective strategy. Our findings emphasize the significance of evaluating the calibration of estimated certainties for model reliability and decision-making.