# GradStop: Exploring Training Dynamics in Unsupervised Outlier Detection through Gradient

arXiv: https://arxiv.org/abs/2412.08501 · HTML: https://mlsanity.com/paper/2412.08501/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Builds on (most cited first, 18 of 18)

- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters](https://mlsanity.com/paper/2008.08692/) (2020)
- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions](https://mlsanity.com/paper/2201.00382/) (2022)
- [Understanding and Improving Early Stopping for Learning with Noisy Labels](https://mlsanity.com/paper/2106.15853/) (2021)
- [Neural Transformation Learning for Deep Anomaly Detection Beyond Images](https://mlsanity.com/paper/2103.16440/) (2021)
- [BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs](https://mlsanity.com/paper/2206.10071/) (2022)
- [Generative Adversarial Active Learning for Unsupervised Outlier Detection](https://mlsanity.com/paper/1809.10816/) (2018)
- [Latent Outlier Exposure for Anomaly Detection with Contaminated Data](https://mlsanity.com/paper/2202.08088/) (2022)
- [Unsupervised Representation Learning by Predicting Random Distances](https://mlsanity.com/paper/1912.12186/) (2019)
- [Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution](https://mlsanity.com/paper/2206.07647/) (2022)
- [A Theoretical Analysis of the Learning Dynamics under Class Imbalance](https://mlsanity.com/paper/2207.00391/) (2022)
- [ODIM: Outlier Detection via Likelihood of Under-Fitted Generative Models](https://mlsanity.com/paper/2301.04257/) (2023)
- [EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy](https://mlsanity.com/paper/2405.12502/) (2024)

## Built on by (most cited first, 3 of 3)

- [INO-SGD: Addressing Utility Imbalance under Individualized Differential Privacy](https://mlsanity.com/paper/2605.07930/) (2026)
- [What Drives the Inlier-Memorization Effect? A Theory of Outlier Detection via Early Training Dynamics](https://mlsanity.com/paper/2606.29791/) (2026)
- [Mitigating Convergence Collapse in Fixed-Target Anomaly Detectors via Kernel-Anchored Locality Regularization](https://mlsanity.com/paper/2610.02345/) (2026)

## Abstract (arXiv)

Unsupervised Outlier Detection (UOD) is a critical task in data mining and machine learning, aiming to identify instances that significantly deviate from the majority. Without any label, deep UOD methods struggle with the misalignment between the model's direct optimization goal and the final performance goal of Outlier Detection (OD) task. Through the perspective of training dynamics, this paper proposes an early stopping algorithm to optimize the training of deep UOD models, ensuring they perform optimally in OD rather than overfitting the entire contaminated dataset. Inspired by UOD mechanism and inlier priority phenomenon, where intuitively models fit inliers more quickly than outliers, we propose GradStop, a sampling-based label-free algorithm to estimate model's real-time performance during training. First, a sampling method generates two sets: one likely containing more outliers and the other more inliers, then a metric based on gradient cohesion is applied to probe into current training dynamics, which reflects model's performance on OD task. Experimental results on 4 deep UOD algorithms and 47 real-world datasets and theoretical proofs demonstrate the effectiveness of our proposed early stopping algorithm in enhancing the performance of deep UOD models. Auto Encoder (AE) enhanced by GradStop achieves better performance than itself, other SOTA UOD methods, and even ensemble AEs. Our method provides a robust and effective solution to the problem of performance degradation during training, enabling deep UOD models to achieve better potential in anomaly detection tasks.