# A Meta-Analysis of the Anomaly Detection Problem

arXiv: https://arxiv.org/abs/1503.01158 · HTML: https://mlsanity.com/paper/1503.01158/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2015.

## Built on by (most cited first, 23 of 23)

- [Deep Isolation Forest for Anomaly Detection](https://mlsanity.com/paper/2206.06602/) (2022)
- [RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection](https://mlsanity.com/paper/2209.08590/) (2022)
- [Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Images](https://mlsanity.com/paper/2205.11474/) (2022)
- [TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models](https://mlsanity.com/paper/2402.10802/) (2024)
- [OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data](https://mlsanity.com/paper/2107.08943/) (2021)
- [A Survey of Graph-based Deep Learning for Anomaly Detection in Distributed Systems](https://mlsanity.com/paper/2206.04149/) (2022)
- [Unsupervised anomaly detection algorithms on real-world data: how many do we need?](https://mlsanity.com/paper/2305.00735/) (2023)
- [gen2Out: Detecting and Ranking Generalized Anomalies](https://mlsanity.com/paper/2109.02704/) (2021)
- [Homophily Outlier Detection in Non-IID Categorical Data](https://mlsanity.com/paper/2103.11516/) (2021)
- [MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection](https://mlsanity.com/paper/2602.09329/) (2026)
- [Data Poisoning Attacks on Off-Policy Policy Evaluation Methods](https://mlsanity.com/paper/2404.04714/) (2024)
- [Oracle Analysis of Representations for Deep Open Set Detection](https://mlsanity.com/paper/2209.11350/) (2022)
- [Detecting Faults during Automatic Screwdriving: A Dataset and Use Case of Anomaly Detection for Automatic Screwdriving](https://mlsanity.com/paper/2107.01955/) (2021)
- [Semi-supervised anomaly detection algorithm based on KL divergence (SAD-KL)](https://mlsanity.com/paper/2203.14539/) (2022)
- [Is AUC the best measure for practical comparison of anomaly detectors?](https://mlsanity.com/paper/2305.04754/) (2023)
- [Unsupervised Outlier Detection using Random Subspace and Subsampling Ensembles of Dirichlet Process Mixtures](https://mlsanity.com/paper/2401.00773/) (2024)
- [Outlier Detection Bias Busted: Understanding Sources of Algorithmic Bias through Data-centric Factors](https://mlsanity.com/paper/2408.13667/) (2024)
- [RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detection](https://mlsanity.com/paper/2311.13959/) (2023)
- [A Non-Parametric Subspace Analysis Approach with Application to Anomaly Detection Ensembles](https://mlsanity.com/paper/2101.04932/) (2021)
- [Anomaly Mining -- Past, Present and Future](https://mlsanity.com/paper/2105.10077/) (2021)
- [Understanding Time Series Anomaly State Detection through One-Class Classification](https://mlsanity.com/paper/2402.02007/) (2024)
- [Detecting Anomalies Using Rotated Isolation Forest](https://mlsanity.com/paper/2501.17787/) (2025)
- [Stylometric Defenses Against Author Impersonation in Software Repositories](https://mlsanity.com/paper/2608.02695/) (2026)

## Abstract (arXiv)

This article provides a thorough meta-analysis of the anomaly detection problem. To accomplish this we first identify approaches to benchmarking anomaly detection algorithms across the literature and produce a large corpus of anomaly detection benchmarks that vary in their construction across several dimensions we deem important to real-world applications: (a) point difficulty, (b) relative frequency of anomalies, (c) clusteredness of anomalies, and (d) relevance of features. We apply a representative set of anomaly detection algorithms to this corpus, yielding a very large collection of experimental results. We analyze these results to understand many phenomena observed in previous work. First we observe the effects of experimental design on experimental results. Second, results are evaluated with two metrics, ROC Area Under the Curve and Average Precision. We employ statistical hypothesis testing to demonstrate the value (or lack thereof) of our benchmarks. We then offer several approaches to summarizing our experimental results, drawing several conclusions about the impact of our methodology as well as the strengths and weaknesses of some algorithms. Last, we compare results against a trivial solution as an alternate means of normalizing the reported performance of algorithms. The intended contributions of this article are many; in addition to providing a large publicly-available corpus of anomaly detection benchmarks, we provide an ontology for describing anomaly detection contexts, a methodology for controlling various aspects of benchmark creation, guidelines for future experimental design and a discussion of the many potential pitfalls of trying to measure success in this field.