# Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN

arXiv: https://arxiv.org/abs/2305.01910 · HTML: https://mlsanity.com/paper/2305.01910/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 3 May 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Deep Label Distribution Learning with Label Ambiguity](https://mlsanity.com/paper/1611.01731/) (2016)
- [TensorMask: A Foundation for Dense Object Segmentation](https://mlsanity.com/paper/1903.12174/) (2019)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth](https://mlsanity.com/paper/1906.11109/) (2019)
- [Deep Watershed Transform for Instance Segmentation](https://mlsanity.com/paper/1611.08303/) (2016)
- [BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors](https://mlsanity.com/paper/1903.03838/) (2019)
- [Probabilistic Object Detection: Definition and Evaluation](https://mlsanity.com/paper/1811.10800/) (2018)
- [Pixelwise Instance Segmentation with a Dynamically Instantiated Network](https://mlsanity.com/paper/1704.02386/) (2017)
- [Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation](https://mlsanity.com/paper/2012.00985/) (2020)
- [Probabilistic Deep Learning for Instance Segmentation](https://mlsanity.com/paper/2008.10678/) (2020)

## Built on by (most cited first, 1 of 1)

- [Self-Supervised Instance Segmentation by Grasping](https://mlsanity.com/paper/2305.06305/) (2023)

## Abstract (arXiv)

Object recognition and instance segmentation are fundamental skills in any robotic or autonomous system. Existing state-of-the-art methods are often unable to capture meaningful uncertainty in challenging or ambiguous scenes, and as such can cause critical errors in high-performance applications. In this paper, we explore a class of distributional instance segmentation models using latent codes that can model uncertainty over plausible hypotheses of object masks. For robotic picking applications, we propose a confidence mask method to achieve the high precision necessary in industrial use cases. We show that our method can significantly reduce critical errors in robotic systems, including our newly released dataset of ambiguous scenes in a robotic application. On a real-world apparel-picking robot, our method significantly reduces double pick errors while maintaining high performance.