# ProPanDL: A Modular Architecture for Uncertainty-Aware Panoptic Segmentation

arXiv: https://arxiv.org/abs/2304.08645 · HTML: https://mlsanity.com/paper/2304.08645/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Evaluating Bayesian Deep Learning Methods for Semantic Segmentation](https://mlsanity.com/paper/1811.12709/) (2018)
- [The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation](https://mlsanity.com/paper/1904.03215/) (2019)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving](https://mlsanity.com/paper/2011.10671/) (2020)
- [Seamless Scene Segmentation](https://mlsanity.com/paper/1905.01220/) (2019)
- [EfficientPS: Efficient Panoptic Segmentation](https://mlsanity.com/paper/2004.02307/) (2020)
- [Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation](https://mlsanity.com/paper/2103.14962/) (2021)
- [Uncertainty-guided Continual Learning with Bayesian Neural Networks](https://mlsanity.com/paper/1906.02425/) (2019)
- [BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors](https://mlsanity.com/paper/1903.03838/) (2019)
- [Probabilistic Object Detection: Definition and Evaluation](https://mlsanity.com/paper/1811.10800/) (2018)
- [Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors](https://mlsanity.com/paper/2101.05036/) (2021)
- [Deep Deterministic Uncertainty for Semantic Segmentation](https://mlsanity.com/paper/2111.00079/) (2021)
- [Uncertainty-aware Panoptic Segmentation](https://mlsanity.com/paper/2206.14554/) (2022)
- [Probabilistic Deep Learning for Instance Segmentation](https://mlsanity.com/paper/2008.10678/) (2020)
- [Estimating Regression Predictive Distributions with Sample Networks](https://mlsanity.com/paper/2211.13724/) (2022)

## Built on by (most cited first, 2 of 2)

- [Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification](https://mlsanity.com/paper/2604.06032/) (2026)
- [U-SEG: Uncertainty in SEGmentation -- A systematic multi-variable exploration](https://mlsanity.com/paper/2605.15421/) (2026)

## Abstract (arXiv)

We introduce ProPanDL, a family of networks capable of uncertainty-aware panoptic segmentation. Unlike existing segmentation methods, ProPanDL is capable of estimating full probability distributions for both the semantic and spatial aspects of panoptic segmentation. We implement and evaluate ProPanDL variants capable of estimating both parametric (Variance Network) and parameter-free (SampleNet) distributions quantifying pixel-wise spatial uncertainty. We couple these approaches with two methods (Temperature Scaling and Evidential Deep Learning) for semantic uncertainty estimation. To evaluate the uncertainty-aware panoptic segmentation task, we address limitations with existing approaches by proposing new metrics that enable separate evaluation of spatial and semantic uncertainty. We additionally propose the use of the energy score, a proper scoring rule, for more robust evaluation of spatial output distributions. Using these metrics, we conduct an extensive evaluation of ProPanDL variants. Our results demonstrate that ProPanDL is capable of estimating well-calibrated and meaningful output distributions while still retaining strong performance on the base panoptic segmentation task.