# Normal Learning in Videos with Attention Prototype Network

arXiv: https://arxiv.org/abs/2108.11055 · HTML: https://mlsanity.com/paper/2108.11055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 54 references that are in the map.
- First posted on arXiv 25 Aug 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Unrolled Generative Adversarial Networks](https://mlsanity.com/paper/1611.02163/) (2016)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection](https://mlsanity.com/paper/1903.07256/) (2019)
- [Latent Space Autoregression for Novelty Detection](https://mlsanity.com/paper/1807.01653/) (2018)
- [Anomaly Detection in Video Sequence with Appearance-Motion Correspondence](https://mlsanity.com/paper/1908.06351/) (2019)
- [Learning a Discriminative Feature Network for Semantic Segmentation](https://mlsanity.com/paper/1804.09337/) (2018)
- [Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection](https://mlsanity.com/paper/2003.06780/) (2020)
- [Learning Normal Dynamics in Videos with Meta Prototype Network](https://mlsanity.com/paper/2104.06689/) (2021)
- [Few-shot Scene-adaptive Anomaly Detection](https://mlsanity.com/paper/2007.07843/) (2020)
- [Abnormal Event Detection in Videos using Spatiotemporal Autoencoder](https://mlsanity.com/paper/1701.01546/) (2017)
- [Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge](https://mlsanity.com/paper/1709.09121/) (2017)

## Abstract (arXiv)

Frame reconstruction (current or future frame) based on Auto-Encoder (AE) is a popular method for video anomaly detection. With models trained on the normal data, the reconstruction errors of anomalous scenes are usually much larger than those of normal ones. Previous methods introduced the memory bank into AE, for encoding diverse normal patterns across the training videos. However, they are memory consuming and cannot cope with unseen new scenarios in the testing data. In this work, we propose a self-attention prototype unit (APU) to encode the normal latent space as prototypes in real time, free from extra memory cost. In addition, we introduce circulative attention mechanism to our backbone to form a novel feature extracting learner, namely Circulative Attention Unit (CAU). It enables the fast adaption capability on new scenes by only consuming a few iterations of update. Extensive experiments are conducted on various benchmarks. The superior performance over the state-of-the-art demonstrates the effectiveness of our method. Our code is available at https://github.com/huchao-AI/APN/.