# Efficient Uncertainty Estimation for Semantic Segmentation in Videos

arXiv: https://arxiv.org/abs/1807.11037 · HTML: https://mlsanity.com/paper/1807.11037/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2018.

## Built on by (most cited first, 30 of 46)

- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild](https://mlsanity.com/paper/2405.18715/) (2024)
- [Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression](https://mlsanity.com/paper/2103.13629/) (2021)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Corner Cases for Visual Perception in Automated Driving: Some Guidance on Detection Approaches](https://mlsanity.com/paper/2102.05897/) (2021)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [Domain Adaptive Video Segmentation via Temporal Consistency Regularization](https://mlsanity.com/paper/2107.11004/) (2021)
- [Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splatting](https://mlsanity.com/paper/2504.01957/) (2025)
- [Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking](https://mlsanity.com/paper/2210.16674/) (2022)
- [RankED: Addressing Imbalance and Uncertainty in Edge Detection Using Ranking-based Losses](https://mlsanity.com/paper/2403.01795/) (2024)
- [Uncertainty-aware Panoptic Segmentation](https://mlsanity.com/paper/2206.14554/) (2022)
- [Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation](https://mlsanity.com/paper/2110.10563/) (2021)
- [Domain Adaptive Video Segmentation via Temporal Pseudo Supervision](https://mlsanity.com/paper/2207.02372/) (2022)
- [Trust It or Not: Confidence-Guided Automatic Radiology Report Generation](https://mlsanity.com/paper/2106.10887/) (2021)
- [Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior](https://mlsanity.com/paper/2306.11739/) (2023)
- [Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Space](https://mlsanity.com/paper/2404.08923/) (2024)
- [Temporal-aware Hierarchical Mask Classification for Video Semantic Segmentation](https://mlsanity.com/paper/2309.08020/) (2023)
- [A Unified Approach Towards Active Learning and Out-of-Distribution Detection](https://mlsanity.com/paper/2405.11337/) (2024)
- [Joint Out-of-Distribution Filtering and Data Discovery Active Learning](https://mlsanity.com/paper/2503.02491/) (2025)
- [Uncertainty-aware LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2210.04472/) (2022)
- [Stream-based Active Learning by Exploiting Temporal Properties in Perception with Temporal Predicted Loss](https://mlsanity.com/paper/2309.05517/) (2023)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [Multimodal Detection of Unknown Objects on Roads for Autonomous Driving](https://mlsanity.com/paper/2205.01414/) (2022)
- [Uncertainty in Real-Time Semantic Segmentation on Embedded Systems](https://mlsanity.com/paper/2301.01201/) (2022)
- [Motion-state Alignment for Video Semantic Segmentation](https://mlsanity.com/paper/2304.08820/) (2023)
- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation](https://mlsanity.com/paper/2604.10950/) (2026)
- [FoV-Net: Field-of-View Extrapolation Using Self-Attention and Uncertainty](https://mlsanity.com/paper/2204.01267/) (2022)
- [Quantifying Model Uncertainty for Semantic Segmentation using Operators in the RKHS](https://mlsanity.com/paper/2211.01999/) (2022)

## Abstract (arXiv)

Uncertainty estimation in deep learning becomes more important recently. A deep learning model can't be applied in real applications if we don't know whether the model is certain about the decision or not. Some literature proposes the Bayesian neural network which can estimate the uncertainty by Monte Carlo Dropout (MC dropout). However, MC dropout needs to forward the model N times which results in N times slower. For real-time applications such as a self-driving car system, which needs to obtain the prediction and the uncertainty as fast as possible, so that MC dropout becomes impractical. In this work, we propose the region-based temporal aggregation (RTA) method which leverages the temporal information in videos to simulate the sampling procedure. Our RTA method with Tiramisu backbone is 10x faster than the MC dropout with Tiramisu backbone (N=5). Furthermore, the uncertainty estimation obtained by our RTA method is comparable to MC dropout's uncertainty estimation on pixel-level and frame-level metrics.