# Non-Parametric Outlier Synthesis

arXiv: https://arxiv.org/abs/2303.02966 · HTML: https://mlsanity.com/paper/2303.02966/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 62 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [ViM: Out-Of-Distribution with Virtual-logit Matching](https://mlsanity.com/paper/2203.10807/) (2022)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [VOS: Learning What You Don't Know by Virtual Outlier Synthesis](https://mlsanity.com/paper/2202.01197/) (2022)
- [Exploring the Limits of Out-of-Distribution Detection](https://mlsanity.com/paper/2106.03004/) (2021)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)

## Built on by (most cited first, 30 of 129)

- [OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection](https://mlsanity.com/paper/2306.09301/) (2023)
- [LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning](https://mlsanity.com/paper/2306.01293/) (2023)
- [ARGS: Alignment as Reward-Guided Search](https://mlsanity.com/paper/2402.01694/) (2024)
- [Process Reward Model with Q-Value Rankings](https://mlsanity.com/paper/2410.11287/) (2024)
- [Dream the Impossible: Outlier Imagination with Diffusion Models](https://mlsanity.com/paper/2309.13415/) (2023)
- [Negative Label Guided OOD Detection with Pretrained Vision-Language Models](https://mlsanity.com/paper/2403.20078/) (2024)
- [Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies](https://mlsanity.com/paper/2503.08558/) (2025)
- [Fast Decision Boundary based Out-of-Distribution Detector](https://mlsanity.com/paper/2312.11536/) (2023)
- [Learning with Mixture of Prototypes for Out-of-Distribution Detection](https://mlsanity.com/paper/2402.02653/) (2024)
- [ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection](https://mlsanity.com/paper/2311.15243/) (2023)
- [Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolation](https://mlsanity.com/paper/2310.13923/) (2023)
- [Learning to Augment Distributions for Out-of-Distribution Detection](https://mlsanity.com/paper/2311.01796/) (2023)
- [GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detection](https://mlsanity.com/paper/2304.04521/) (2023)
- [How Does Unlabeled Data Provably Help Out-of-Distribution Detection?](https://mlsanity.com/paper/2402.03502/) (2024)
- [Outlier detection by ensembling uncertainty with negative objectness](https://mlsanity.com/paper/2402.15374/) (2024)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sources](https://mlsanity.com/paper/2311.03236/) (2023)
- [MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities](https://mlsanity.com/paper/2405.17419/) (2024)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2311.03524/) (2023)
- [LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2407.08966/) (2024)
- [OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learning](https://mlsanity.com/paper/2402.04129/) (2024)
- [Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization](https://mlsanity.com/paper/2401.15914/) (2024)
- [Reliable Few-shot Learning under Dual Noises](https://mlsanity.com/paper/2506.16330/) (2025)
- [AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2410.20149/) (2024)
- [Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection](https://mlsanity.com/paper/2409.04796/) (2024)
- [Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection](https://mlsanity.com/paper/2411.03359/) (2024)
- [GOLD: Graph Out-of-Distribution Detection via Implicit Adversarial Latent Generation](https://mlsanity.com/paper/2502.05780/) (2025)
- [NNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation](https://mlsanity.com/paper/2311.11961/) (2023)

## Abstract (arXiv)

Out-of-distribution (OOD) detection is indispensable for safely deploying machine learning models in the wild. One of the key challenges is that models lack supervision signals from unknown data, and as a result, can produce overconfident predictions on OOD data. Recent work on outlier synthesis modeled the feature space as parametric Gaussian distribution, a strong and restrictive assumption that might not hold in reality. In this paper, we propose a novel framework, Non-Parametric Outlier Synthesis (NPOS), which generates artificial OOD training data and facilitates learning a reliable decision boundary between ID and OOD data. Importantly, our proposed synthesis approach does not make any distributional assumption on the ID embeddings, thereby offering strong flexibility and generality. We show that our synthesis approach can be mathematically interpreted as a rejection sampling framework. Extensive experiments show that NPOS can achieve superior OOD detection performance, outperforming the competitive rivals by a significant margin. Code is publicly available at https://github.com/deeplearning-wisc/npos.