# Benchmarking Unsupervised Outlier Detection with Realistic Synthetic Data

arXiv: https://arxiv.org/abs/2004.06947 · HTML: https://mlsanity.com/paper/2004.06947/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 25 of 25)

- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs](https://mlsanity.com/paper/2206.10071/) (2022)
- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [Unsupervised anomaly detection algorithms on real-world data: how many do we need?](https://mlsanity.com/paper/2305.00735/) (2023)
- [Toward Unsupervised Outlier Model Selection](https://mlsanity.com/paper/2211.01834/) (2022)
- [ADGym: Design Choices for Deep Anomaly Detection](https://mlsanity.com/paper/2309.15376/) (2023)
- [Anomaly Detection with Score Distribution Discrimination](https://mlsanity.com/paper/2306.14403/) (2023)
- [TACTIC for Navigating the Unknown: Tabular Anomaly deteCTion via In-Context inference](https://mlsanity.com/paper/2603.14171/) (2026)
- [TAD-Bench: A Comprehensive Benchmark for Embedding-Based Text Anomaly Detection](https://mlsanity.com/paper/2501.11960/) (2025)
- [MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection](https://mlsanity.com/paper/2602.09329/) (2026)
- [Learning-Based Link Anomaly Detection in Continuous-Time Dynamic Graphs](https://mlsanity.com/paper/2405.18050/) (2024)
- [Understanding the limitations of self-supervised learning for tabular anomaly detection](https://mlsanity.com/paper/2309.08374/) (2023)
- [Fuzzy Granule Density-Based Outlier Detection with Multi-Scale Granular Balls](https://mlsanity.com/paper/2501.02975/) (2025)
- [No Pattern, No Recognition: a Survey about Reproducibility and Distortion Issues of Text Clustering and Topic Modeling](https://mlsanity.com/paper/2208.01712/) (2022)
- [From Explanation to Action: An End-to-End Human-in-the-loop Framework for Anomaly Reasoning and Management](https://mlsanity.com/paper/2304.03368/) (2023)
- [C-AllOut: Catching & Calling Outliers by Type](https://mlsanity.com/paper/2110.08257/) (2021)
- [Benchmarking Anomaly Detection Algorithms: Deep Learning and Beyond](https://mlsanity.com/paper/2402.07281/) (2024)
- [Unsupervised Parameter-free Outlier Detection using HDBSCAN* Outlier Profiles](https://mlsanity.com/paper/2411.08867/) (2024)
- [AD-MERCS: Modeling Normality and Abnormality in Unsupervised Anomaly Detection](https://mlsanity.com/paper/2305.12958/) (2023)
- [Unleashing the Potential of Unsupervised Deep Outlier Detection through Automated Training Stopping](https://mlsanity.com/paper/2305.16777/) (2023)
- [Hyperspectral Unmixing Under Endmember Variability: A Variational Inference Framework](https://mlsanity.com/paper/2407.14899/) (2024)
- [Real-Time Anomaly Detection with Synthetic Anomaly Monitoring (SAM)](https://mlsanity.com/paper/2501.18417/) (2025)
- [Investigating Mask-aware Prototype Learning for Tabular Anomaly Detection](https://mlsanity.com/paper/2506.02757/) (2025)
- [Real-Time Decorrelation-Based Anomaly Detection for Multivariate Time Series](https://mlsanity.com/paper/2507.07559/) (2025)
- [Why the Counterintuitive Phenomenon of Likelihood Rarely Appears in Tabular Anomaly Detection with Deep Generative Models?](https://mlsanity.com/paper/2602.09593/) (2026)

## Abstract (arXiv)

Benchmarking unsupervised outlier detection is difficult. Outliers are rare, and existing benchmark data contains outliers with various and unknown characteristics. Fully synthetic data usually consists of outliers and regular instance with clear characteristics and thus allows for a more meaningful evaluation of detection methods in principle. Nonetheless, there have only been few attempts to include synthetic data in benchmarks for outlier detection. This might be due to the imprecise notion of outliers or to the difficulty to arrive at a good coverage of different domains with synthetic data. In this work we propose a generic process for the generation of data sets for such benchmarking. The core idea is to reconstruct regular instances from existing real-world benchmark data while generating outliers so that they exhibit insightful characteristics. This allows both for a good coverage of domains and for helpful interpretations of results. We also describe three instantiations of the generic process that generate outliers with specific characteristics, like local outliers. A benchmark with state-of-the-art detection methods confirms that our generic process is indeed practical.