# Diminishing Empirical Risk Minimization for Unsupervised Anomaly Detection

arXiv: https://arxiv.org/abs/2205.14676 · HTML: https://mlsanity.com/paper/2205.14676/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 16 of 16)

- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications](https://mlsanity.com/paper/1802.03903/) (2018)
- [PyOD: A Python Toolbox for Scalable Outlier Detection](https://mlsanity.com/paper/1901.01588/) (2019)
- [A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data](https://mlsanity.com/paper/1811.08055/) (2018)
- [Tilted Empirical Risk Minimization](https://mlsanity.com/paper/2007.01162/) (2020)
- [Generative Adversarial Active Learning for Unsupervised Outlier Detection](https://mlsanity.com/paper/1809.10816/) (2018)
- [Learning From Noisy Singly-labeled Data](https://mlsanity.com/paper/1712.04577/) (2017)
- [Biased Importance Sampling for Deep Neural Network Training](https://mlsanity.com/paper/1706.00043/) (2017)
- [Robust Subspace Recovery Layer for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.00152/) (2019)
- [Anomaly Detection using Autoencoders in High Performance Computing Systems](https://mlsanity.com/paper/1811.05269/) (2018)
- [Robust, Deep and Inductive Anomaly Detection](https://mlsanity.com/paper/1704.06743/) (2017)
- [Clustering and Unsupervised Anomaly Detection with L2 Normalized Deep Auto-Encoder Representations](https://mlsanity.com/paper/1802.00187/) (2018)

## Abstract (arXiv)

Unsupervised anomaly detection (AD) is a challenging task in realistic applications. Recently, there is an increasing trend to detect anomalies with deep neural networks (DNN). However, most popular deep AD detectors cannot protect the network from learning contaminated information brought by anomalous data, resulting in unsatisfactory detection performance and overfitting issues. In this work, we identify one reason that hinders most existing DNN-based anomaly detection methods from performing is the wide adoption of the Empirical Risk Minimization (ERM). ERM assumes that the performance of an algorithm on an unknown distribution can be approximated by averaging losses on the known training set. This averaging scheme thus ignores the distinctions between normal and anomalous instances. To break through the limitations of ERM, we propose a novel Diminishing Empirical Risk Minimization (DERM) framework. Specifically, DERM adaptively adjusts the impact of individual losses through a well-devised aggregation strategy. Theoretically, our proposed DERM can directly modify the gradient contribution of each individual loss in the optimization process to suppress the influence of outliers, leading to a robust anomaly detector. Empirically, DERM outperformed the state-of-the-art on the unsupervised AD benchmark consisting of 18 datasets.