# Deep Structured Energy Based Models for Anomaly Detection

arXiv: https://arxiv.org/abs/1605.07717 · HTML: https://mlsanity.com/paper/1605.07717/

## Key facts

- Cited by 114 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2016.

## Built on by (most cited first, 30 of 114)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection](https://mlsanity.com/paper/2110.02855/) (2021)
- [Asymmetric Student-Teacher Networks for Industrial Anomaly Detection](https://mlsanity.com/paper/2210.07829/) (2022)
- [A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection](https://mlsanity.com/paper/2104.14535/) (2021)
- [Video Event Restoration Based on Keyframes for Video Anomaly Detection](https://mlsanity.com/paper/2304.05112/) (2023)
- [Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2404.08531/) (2024)
- [Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Need](https://mlsanity.com/paper/2302.02615/) (2023)
- [Unsupervised Out-of-Domain Detection via Pre-trained Transformers](https://mlsanity.com/paper/2106.00948/) (2021)
- [Dynamic Local Aggregation Network with Adaptive Clusterer for Anomaly Detection](https://mlsanity.com/paper/2207.10948/) (2022)
- [A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Images](https://mlsanity.com/paper/2204.11161/) (2022)
- [Visual Anomaly Detection Via Partition Memory Bank Module and Error Estimation](https://mlsanity.com/paper/2209.12441/) (2022)
- [Adaptive Memory Networks with Self-supervised Learning for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2201.00464/) (2022)
- [On Out-of-distribution Detection with Energy-based Models](https://mlsanity.com/paper/2107.08785/) (2021)
- [Deep Learning Advancements in Anomaly Detection: A Comprehensive Survey](https://mlsanity.com/paper/2503.13195/) (2025)
- [Robust and Scalable Bayesian Online Changepoint Detection](https://mlsanity.com/paper/2302.04759/) (2023)
- [Graph-level Anomaly Detection via Hierarchical Memory Networks](https://mlsanity.com/paper/2307.00755/) (2023)
- [Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2110.12076/) (2021)
- [A survey and taxonomy of loss functions in machine learning](https://mlsanity.com/paper/2301.05579/) (2023)
- [OLED: One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection](https://mlsanity.com/paper/2103.14953/) (2021)
- [Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Results](https://mlsanity.com/paper/2107.04982/) (2021)
- [Stabilizing Adversarially Learned One-Class Novelty Detection Using Pseudo Anomalies](https://mlsanity.com/paper/2203.13716/) (2022)
- [Adaptive Model Pooling for Online Deep Anomaly Detection from a Complex Evolving Data Stream](https://mlsanity.com/paper/2206.04792/) (2022)
- [MemStream: Memory-Based Streaming Anomaly Detection](https://mlsanity.com/paper/2106.03837/) (2021)
- [Unsupervised Outlier Detection using Memory and Contrastive Learning](https://mlsanity.com/paper/2107.12642/) (2021)
- [A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms](https://mlsanity.com/paper/2204.09825/) (2022)
- [Anomaly Detection by Leveraging Incomplete Anomalous Knowledge with Anomaly-Aware Bidirectional GANs](https://mlsanity.com/paper/2204.13335/) (2022)
- [Anomaly Detection in Aerial Videos with Transformers](https://mlsanity.com/paper/2209.13363/) (2022)
- [CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning](https://mlsanity.com/paper/2210.01742/) (2022)

## Abstract (arXiv)

In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching citesm, which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods.