# Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution

arXiv: https://arxiv.org/abs/2206.07647 · HTML: https://mlsanity.com/paper/2206.07647/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 53 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [Hyperparameter Ensembles for Robustness and Uncertainty Quantification](https://mlsanity.com/paper/2006.13570/) (2020)
- [Efficient GAN-Based Anomaly Detection](https://mlsanity.com/paper/1802.06222/) (2018)
- [DROCC: Deep Robust One-Class Classification](https://mlsanity.com/paper/2002.12718/) (2020)
- [Neural Transformation Learning for Deep Anomaly Detection Beyond Images](https://mlsanity.com/paper/2103.16440/) (2021)
- [How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms?](https://mlsanity.com/paper/1607.01152/) (2016)

## Built on by (most cited first, 24 of 24)

- [BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs](https://mlsanity.com/paper/2206.10071/) (2022)
- [ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels](https://mlsanity.com/paper/2208.11290/) (2022)
- [Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Success](https://mlsanity.com/paper/2208.07734/) (2022)
- [Robust One-Class Classification with Signed Distance Function using 1-Lipschitz Neural Networks](https://mlsanity.com/paper/2303.01978/) (2023)
- [From Zero to Hero: Advancing Zero-Shot Foundation Models for Tabular Outlier Detection](https://mlsanity.com/paper/2602.03018/) (2026)
- [Model Selection of Anomaly Detectors in the Absence of Labeled Validation Data](https://mlsanity.com/paper/2310.10461/) (2023)
- [On the Detection of Reviewer-Author Collusion Rings From Paper Bidding](https://mlsanity.com/paper/2402.07860/) (2024)
- [Fast Unsupervised Deep Outlier Model Selection with Hypernetworks](https://mlsanity.com/paper/2307.10529/) (2023)
- [D.MCA: Outlier Detection with Explicit Micro-Cluster Assignments](https://mlsanity.com/paper/2210.08212/) (2022)
- [MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection](https://mlsanity.com/paper/2602.09329/) (2026)
- [Hyperparameter Optimization for Unsupervised Outlier Detection](https://mlsanity.com/paper/2208.11727/) (2022)
- [EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy](https://mlsanity.com/paper/2405.12502/) (2024)
- [We Need to Rethink Benchmarking in Anomaly Detection](https://mlsanity.com/paper/2507.15584/) (2025)
- [GradStop: Exploring Training Dynamics in Unsupervised Outlier Detection through Gradient](https://mlsanity.com/paper/2412.08501/) (2024)
- [Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities](https://mlsanity.com/paper/2308.14380/) (2023)
- [Outlier Detection Bias Busted: Understanding Sources of Algorithmic Bias through Data-centric Factors](https://mlsanity.com/paper/2408.13667/) (2024)
- [Unleashing the Potential of Unsupervised Deep Outlier Detection through Automated Training Stopping](https://mlsanity.com/paper/2305.16777/) (2023)
- [Towards Automated Self-Supervised Learning for Truly Unsupervised Graph Anomaly Detection](https://mlsanity.com/paper/2501.14694/) (2025)
- [UniOD: A Universal Model for Outlier Detection across Diverse Domains](https://mlsanity.com/paper/2507.06624/) (2025)
- [Automatic Unsupervised Ensemble Outlier Model Selection--Extended Version](https://mlsanity.com/paper/2605.16567/) (2026)
- [What Drives the Inlier-Memorization Effect? A Theory of Outlier Detection via Early Training Dynamics](https://mlsanity.com/paper/2606.29791/) (2026)
- [InFlux++: Real and Synthetic Data for Estimating Dynamic Camera Intrinsics](https://mlsanity.com/paper/2607.05389/) (2026)
- [Unsupervised Anomaly Detection Using Flow Matching on Tabular Data](https://mlsanity.com/paper/2608.19801/) (2026)
- [TaskBridge: Bridging Unsupervised Tabular Anomaly Detection and In-Context Learning via Virtual Tasks](https://mlsanity.com/paper/2609.36968/) (2026)

## Abstract (arXiv)

Outlier detection (OD) literature exhibits numerous algorithms as it applies to diverse domains. However, given a new detection task, it is unclear how to choose an algorithm to use, nor how to set its hyperparameter(s) (HPs) in unsupervised settings. HP tuning is an ever-growing problem with the arrival of many new detectors based on deep learning, which usually come with a long list of HPs. Surprisingly, the issue of model selection in the outlier mining literature has been "the elephant in the room"; a significant factor in unlocking the utmost potential of deep methods, yet little said or done to systematically tackle the issue. In the first part of this paper, we conduct the first large-scale analysis on the HP sensitivity of deep OD methods, and through more than 35,000 trained models, quantitatively demonstrate that model selection is inevitable. Next, we design a HP-robust and scalable deep hyper-ensemble model called ROBOD that assembles models with varying HP configurations, bypassing the choice paralysis. Importantly, we introduce novel strategies to speed up ensemble training, such as parameter sharing, batch/simultaneous training, and data subsampling, that allow us to train fewer models with fewer parameters. Extensive experiments on both image and tabular datasets show that ROBOD achieves and retains robust, state-of-the-art detection performance as compared to its modern counterparts, while taking only 2-10% of the time by the naive hyper-ensemble with independent training.