# False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates

arXiv: https://arxiv.org/abs/2106.14474 · HTML: https://mlsanity.com/paper/2106.14474/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Multivariate Confidence Calibration for Object Detection](https://mlsanity.com/paper/2004.13546/) (2020)
- [Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilities](https://mlsanity.com/paper/1811.00648/) (2018)
- [Leveraging Uncertainty Estimates for Predicting Segmentation Quality](https://mlsanity.com/paper/1807.00502/) (2018)
- [Instance-Level Salient Object Segmentation](https://mlsanity.com/paper/1704.03604/) (2017)
- [Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks](https://mlsanity.com/paper/1911.05075/) (2019)
- [Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates](https://mlsanity.com/paper/2012.07504/) (2020)
- [Propagating Uncertainty in Multi-Stage Bayesian Convolutional Neural Networks with Application to Pulmonary Nodule Detection](https://mlsanity.com/paper/1712.00497/) (2017)
- [MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation](https://mlsanity.com/paper/1912.07420/) (2019)
- [Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems](https://mlsanity.com/paper/1907.11066/) (2019)
- [A Comparative Study of High-Recall Real-Time Semantic Segmentation Based on Swift Factorized Network](https://mlsanity.com/paper/1907.11394/) (2019)
- [False Detection (Positives and Negatives) in Object Detection](https://mlsanity.com/paper/2008.06986/) (2020)

## Built on by (most cited first, 6 of 6)

- [Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects](https://mlsanity.com/paper/2210.02074/) (2022)
- [Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentation](https://mlsanity.com/paper/2303.06920/) (2023)
- [False Negative Reduction in Semantic Segmentation under Domain Shift using Depth Estimation](https://mlsanity.com/paper/2207.03513/) (2022)
- [Plausibility Verification For 3D Object Detectors Using Energy-Based Optimization](https://mlsanity.com/paper/2211.05233/) (2022)
- [Uncertainty and Prediction Quality Estimation for Semantic Segmentation via Graph Neural Networks](https://mlsanity.com/paper/2409.11373/) (2024)
- [U-SEG: Uncertainty in SEGmentation -- A systematic multi-variable exploration](https://mlsanity.com/paper/2605.15421/) (2026)

## Abstract (arXiv)

Instance segmentation of images is an important tool for automated scene understanding. Neural networks are usually trained to optimize their overall performance in terms of accuracy. Meanwhile, in applications such as automated driving, an overlooked pedestrian seems more harmful than a falsely detected one. In this work, we present a false negative detection method for image sequences based on inconsistencies in time series of tracked instances given the availability of image sequences in online applications. As the number of instances can be greatly increased by this algorithm, we apply a false positive pruning using uncertainty estimates aggregated over instances. To this end, instance-wise metrics are constructed which characterize uncertainty and geometry of a given instance or are predicated on depth estimation. The proposed method serves as a post-processing step applicable to any neural network that can also be trained on single frames only. In our tests, we obtain an improved trade-off between false negative and false positive instances by our fused detection approach in comparison to the use of an ordinary score value provided by the instance segmentation network during inference.