# Outlier detection by ensembling uncertainty with negative objectness

arXiv: https://arxiv.org/abs/2402.15374 · HTML: https://mlsanity.com/paper/2402.15374/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 85 of its 108 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 30 of 85)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)

## Built on by (most cited first, 30 of 34)

- [PixOOD: Pixel-Level Out-of-Distribution Detection](https://mlsanity.com/paper/2405.19882/) (2024)
- [Out-of-Distribution Segmentation in Autonomous Driving: Problems and State of the Art](https://mlsanity.com/paper/2503.08695/) (2025)
- [Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving](https://mlsanity.com/paper/2505.02148/) (2025)
- [Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation](https://mlsanity.com/paper/2504.04841/) (2025)
- [MC-PanDA: Mask Confidence for Panoptic Domain Adaptation](https://mlsanity.com/paper/2407.14110/) (2024)
- [VL4AD: Vision-Language Models Improve Pixel-wise Anomaly Detection](https://mlsanity.com/paper/2409.17330/) (2024)
- [AnomalyVFM -- Transforming Vision Foundation Models into Zero-Shot Anomaly Detectors](https://mlsanity.com/paper/2601.20524/) (2026)
- [UMAD: Unsupervised Mask-Level Anomaly Detection for Autonomous Driving](https://mlsanity.com/paper/2406.06370/) (2024)
- [Segment-Level Road Obstacle Detection Using Visual Foundation Model Priors and Likelihood Ratios](https://mlsanity.com/paper/2412.05707/) (2024)
- [Multi-Scale Foreground-Background Confidence for Out-of-Distribution Segmentation](https://mlsanity.com/paper/2412.16990/) (2024)
- [Uncertainty-Aware Likelihood Ratio Estimation for Pixel-Wise Out-of-Distribution Detection](https://mlsanity.com/paper/2508.00587/) (2025)
- [FlowCLAS: Enhancing Normalizing Flow Via Contrastive Learning For Anomaly Segmentation](https://mlsanity.com/paper/2411.19888/) (2024)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [A Dataset for Semantic Segmentation in the Presence of Unknowns](https://mlsanity.com/paper/2503.22309/) (2025)
- [Panoramic Out-of-Distribution Segmentation](https://mlsanity.com/paper/2505.03539/) (2025)
- [Out-of-Distribution Semantic Occupancy Prediction](https://mlsanity.com/paper/2506.21185/) (2025)
- [Randomized PCA Forest for Unsupervised Outlier Detection](https://mlsanity.com/paper/2508.12776/) (2025)
- [Relative Energy Learning for LiDAR Out-of-Distribution Detection](https://mlsanity.com/paper/2511.06720/) (2025)
- [Label-Free Model Failure Detection for Lidar-based Point Cloud Segmentation](https://mlsanity.com/paper/2407.14306/) (2024)
- [Open-set Anomaly Segmentation in Complex Scenarios](https://mlsanity.com/paper/2504.19706/) (2025)
- [Out-of-Distribution Segmentation via Wasserstein-Based Evidential Uncertainty](https://mlsanity.com/paper/2512.11373/) (2025)
- [Neural Network Meta Classifier: Improving the Reliability of Anomaly Segmentation](https://mlsanity.com/paper/2412.10765/) (2024)
- [Objectomaly: Objectness-Aware Refinement for OoD Segmentation with Structural Consistency and Boundary Precision](https://mlsanity.com/paper/2507.07460/) (2025)
- [A Machine Learning Perspective on Automated Driving Corner Cases](https://mlsanity.com/paper/2510.10653/) (2025)
- [Language-Guided Open-World Anomaly Segmentation](https://mlsanity.com/paper/2512.01427/) (2025)
- [ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Data](https://mlsanity.com/paper/2512.02686/) (2025)
- [ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Prediction](https://mlsanity.com/paper/2604.01081/) (2026)
- [Neural Distribution Prior for LiDAR Out-of-Distribution Detection](https://mlsanity.com/paper/2604.09232/) (2026)
- [Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentation](https://mlsanity.com/paper/2604.23604/) (2026)
- [Real-World On-Vehicle Evaluation of Embedding-Based Anomaly Detection](https://mlsanity.com/paper/2605.19744/) (2026)

## Abstract (arXiv)

Outlier detection is an essential capability in safety-critical applications of supervised visual recognition. Most of the existing methods deliver best results by encouraging standard closed-set models to produce low-confidence predictions in negative training data. However, that approach conflates prediction uncertainty with recognition of the negative class. We therefore reconsider direct prediction of K+1 logits that correspond to K groundtruth classes and one outlier class. This setup allows us to formulate a novel anomaly score as an ensemble of in-distribution uncertainty and the posterior of the outlier class which we term negative objectness. Now outliers can be independently detected due to i) high prediction uncertainty or ii) similarity with negative data. We embed our method into a dense prediction architecture with mask-level recognition over K+2 classes. The training procedure encourages the novel K+2-th class to learn negative objectness at pasted negative instances. Our models outperform the current state-of-the art on standard benchmarks for image-wide and pixel-level outlier detection with and without training on real negative data.