# Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates

arXiv: https://arxiv.org/abs/2012.07504 · HTML: https://mlsanity.com/paper/2012.07504/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 14 of 14)

- [Probabilistic Representations for Video Contrastive Learning](https://mlsanity.com/paper/2204.03946/) (2022)
- [Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects](https://mlsanity.com/paper/2210.02074/) (2022)
- [Gradient-Based Quantification of Epistemic Uncertainty for Deep Object Detectors](https://mlsanity.com/paper/2107.04517/) (2021)
- [MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification](https://mlsanity.com/paper/2405.19186/) (2024)
- [Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentation](https://mlsanity.com/paper/2303.06920/) (2023)
- [False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates](https://mlsanity.com/paper/2106.14474/) (2021)
- [What is Point Supervision Worth in Video Instance Segmentation?](https://mlsanity.com/paper/2404.01990/) (2024)
- [UVIS: Unsupervised Video Instance Segmentation](https://mlsanity.com/paper/2406.06908/) (2024)
- [False Negative Reduction in Semantic Segmentation under Domain Shift using Depth Estimation](https://mlsanity.com/paper/2207.03513/) (2022)
- [Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models -](https://mlsanity.com/paper/2504.12137/) (2025)
- [Uncertainty and Prediction Quality Estimation for Semantic Segmentation via Graph Neural Networks](https://mlsanity.com/paper/2409.11373/) (2024)
- [Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection](https://mlsanity.com/paper/2506.21486/) (2025)
- [Spatio-Temporal Attention for Consistent Video Semantic Segmentation in Automated Driving](https://mlsanity.com/paper/2602.10052/) (2026)
- [U-SEG: Uncertainty in SEGmentation -- A systematic multi-variable exploration](https://mlsanity.com/paper/2605.15421/) (2026)

## Abstract (arXiv)

Instance segmentation with neural networks is an essential task in environment perception. In many works, it has been observed that neural networks can predict false positive instances with high confidence values and true positives with low ones. Thus, it is important to accurately model the uncertainties of neural networks in order to prevent safety issues and foster interpretability. In applications such as automated driving, the reliability of neural networks is of highest interest. In this paper, we present a time-dynamic approach to model uncertainties of instance segmentation networks and apply this to the detection of false positives as well as the estimation of prediction quality. The availability of image sequences in online applications allows for tracking instances over multiple frames. Based on an instances history of shape and uncertainty information, we construct temporal instance-wise aggregated metrics. The latter are used as input to post-processing models that estimate the prediction quality in terms of instance-wise intersection over union. The proposed method only requires a readily trained neural network (that may operate on single frames) and video sequence input. In our experiments, we further demonstrate the use of the proposed method by replacing the traditional score value from object detection and thereby improving the overall performance of the instance segmentation network.