# Further Analysis of Outlier Detection with Deep Generative Models

arXiv: https://arxiv.org/abs/2010.13064 · HTML: https://mlsanity.com/paper/2010.13064/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2020.

## Built on by (most cited first, 30 of 31)

- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [Understanding Failures in Out-of-Distribution Detection with Deep Generative Models](https://mlsanity.com/paper/2107.06908/) (2021)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection](https://mlsanity.com/paper/2209.08590/) (2022)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [GOLD: Graph Out-of-Distribution Detection via Implicit Adversarial Latent Generation](https://mlsanity.com/paper/2502.05780/) (2025)
- [Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penalty](https://mlsanity.com/paper/2212.12641/) (2022)
- [Foundational World Models Accurately Detect Bimanual Manipulator Failures](https://mlsanity.com/paper/2603.06987/) (2026)
- [Understanding Out-of-distribution: A Perspective of Data Dynamics](https://mlsanity.com/paper/2111.14730/) (2021)
- [Detecting Anomalous Event Sequences with Temporal Point Processes](https://mlsanity.com/paper/2106.04465/) (2021)
- [DSLOB: A Synthetic Limit Order Book Dataset for Benchmarking Forecasting Algorithms under Distributional Shift](https://mlsanity.com/paper/2211.11513/) (2022)
- [Graph Synthetic Out-of-Distribution Exposure with Large Language Models](https://mlsanity.com/paper/2504.21198/) (2025)
- [A Comprehensive Review of Trends, Applications and Challenges In Out-of-Distribution Detection](https://mlsanity.com/paper/2209.12935/) (2022)
- [Understanding Likelihood of Normalizing Flow and Image Complexity through the Lens of Out-of-Distribution Detection](https://mlsanity.com/paper/2402.10477/) (2024)
- [Diffusion-based Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/2411.10701/) (2024)
- [Investigating the Adversarial Robustness of Density Estimation Using the Probability Flow ODE](https://mlsanity.com/paper/2310.07084/) (2023)
- [Your Classifier Can Be Secretly a Likelihood-Based OOD Detector](https://mlsanity.com/paper/2408.04851/) (2024)
- [Out-of-Distribution Detection in Time-Series Domain: A Novel Seasonal Ratio Scoring Approach](https://mlsanity.com/paper/2207.04306/) (2022)
- [Brittle Features May Help Anomaly Detection](https://mlsanity.com/paper/2104.10453/) (2021)
- [Outcome-Guided Counterfactuals for Reinforcement Learning Agents from a Jointly Trained Generative Latent Space](https://mlsanity.com/paper/2207.07710/) (2022)
- [ScatterAD: Temporal-Topological Scattering Mechanism for Time Series Anomaly Detection](https://mlsanity.com/paper/2509.24414/) (2025)
- [RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detection](https://mlsanity.com/paper/2311.13959/) (2023)
- [Advancing Out-of-Distribution Detection through Data Purification and Dynamic Activation Function Design](https://mlsanity.com/paper/2403.03412/) (2024)
- [Outlier-Robust Diffusion Solvers for Inverse Problems](https://mlsanity.com/paper/2605.09477/) (2026)
- [Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization](https://mlsanity.com/paper/2209.12807/) (2022)
- [On the Connection of Generative Models and Discriminative Models for Anomaly Detection](https://mlsanity.com/paper/2211.08910/) (2022)
- [Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/2409.03801/) (2024)

## Abstract (arXiv)

The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling. In this work, we present a possible explanation for this phenomenon, starting from the observation that a model's typical set and high-density region may not conincide. From this vantage point we propose a novel outlier test, the empirical success of which suggests that the failure of existing likelihood-based outlier tests does not necessarily imply that the corresponding generative model is uncalibrated. We also conduct additional experiments to help disentangle the impact of low-level texture versus high-level semantics in differentiating outliers. In aggregate, these results suggest that modifications to the standard evaluation practices and benchmarks commonly applied in the literature are needed.