# Learning Normal Dynamics in Videos with Meta Prototype Network

arXiv: https://arxiv.org/abs/2104.06689 · HTML: https://mlsanity.com/paper/2104.06689/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Unrolled Generative Adversarial Networks](https://mlsanity.com/paper/1611.02163/) (2016)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection](https://mlsanity.com/paper/1903.07256/) (2019)
- [Latent Space Autoregression for Novelty Detection](https://mlsanity.com/paper/1807.01653/) (2018)
- [Anomaly Detection in Video Sequence with Appearance-Motion Correspondence](https://mlsanity.com/paper/1908.06351/) (2019)
- [Learning a Discriminative Feature Network for Semantic Segmentation](https://mlsanity.com/paper/1804.09337/) (2018)
- [Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection](https://mlsanity.com/paper/2003.06780/) (2020)
- [Few-shot Scene-adaptive Anomaly Detection](https://mlsanity.com/paper/2007.07843/) (2020)
- [Abnormal Event Detection in Videos using Spatiotemporal Autoencoder](https://mlsanity.com/paper/1701.01546/) (2017)
- [Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge](https://mlsanity.com/paper/1709.09121/) (2017)
- [Localizing Anomalies from Weakly-Labeled Videos](https://mlsanity.com/paper/2008.08944/) (2020)
- [Unmasking the abnormal events in video](https://mlsanity.com/paper/1705.08182/) (2017)

## Built on by (most cited first, 30 of 71)

- [Harnessing Large Language Models for Training-free Video Anomaly Detection](https://mlsanity.com/paper/2404.01014/) (2024)
- [Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2310.14228/) (2023)
- [Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2303.12369/) (2023)
- [Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2407.10299/) (2024)
- [Video Event Restoration Based on Keyframes for Video Anomaly Detection](https://mlsanity.com/paper/2304.05112/) (2023)
- [Advancing Video Anomaly Detection: A Concise Review and a New Dataset](https://mlsanity.com/paper/2402.04857/) (2024)
- [Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2404.08531/) (2024)
- [Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detection](https://mlsanity.com/paper/2503.02424/) (2025)
- [A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and Anticipation](https://mlsanity.com/paper/2305.13611/) (2023)
- [Diversity-Measurable Anomaly Detection](https://mlsanity.com/paper/2303.05047/) (2023)
- [Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detection](https://mlsanity.com/paper/2303.13051/) (2023)
- [Anomaly Detection under Distribution Shift](https://mlsanity.com/paper/2303.13845/) (2023)
- [SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2111.13495/) (2021)
- [Dynamic Local Aggregation Network with Adaptive Clusterer for Anomaly Detection](https://mlsanity.com/paper/2207.10948/) (2022)
- [Delving into CLIP latent space for Video Anomaly Recognition](https://mlsanity.com/paper/2310.02835/) (2023)
- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)
- [MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection](https://mlsanity.com/paper/2403.14497/) (2024)
- [Deep Anomaly Discovery From Unlabeled Videos via Normality Advantage and Self-Paced Refinement](https://mlsanity.com/paper/2108.01975/) (2021)
- [Cross-Domain Video Anomaly Detection without Target Domain Adaptation](https://mlsanity.com/paper/2212.07010/) (2022)
- [Context Recovery and Knowledge Retrieval: A Novel Two-Stream Framework for Video Anomaly Detection](https://mlsanity.com/paper/2209.02899/) (2022)
- [Learnable Locality-Sensitive Hashing for Video Anomaly Detection](https://mlsanity.com/paper/2111.07839/) (2021)
- [AnyAnomaly: Zero-Shot Customizable Video Anomaly Detection with LVLM](https://mlsanity.com/paper/2503.04504/) (2025)
- [MissionGNN: Hierarchical Multimodal GNN-based Weakly Supervised Video Anomaly Recognition with Mission-Specific Knowledge Graph Generation](https://mlsanity.com/paper/2406.18815/) (2024)
- [Video Anomaly Detection with Motion and Appearance Guided Patch Diffusion Model](https://mlsanity.com/paper/2412.09026/) (2024)
- [Exploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images](https://mlsanity.com/paper/2403.08689/) (2024)
- [Context-aware Video Anomaly Detection in Long-Term Datasets](https://mlsanity.com/paper/2404.07887/) (2024)
- [MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection](https://mlsanity.com/paper/2510.21449/) (2025)
- [SLSG: Industrial Image Anomaly Detection by Learning Better Feature Embeddings and One-Class Classification](https://mlsanity.com/paper/2305.00398/) (2023)
- [REB: Reducing Biases in Representation for Industrial Anomaly Detection](https://mlsanity.com/paper/2308.12577/) (2023)
- [Towards Visual Discrimination and Reasoning of Real-World Physical Dynamics: Physics-Grounded Anomaly Detection](https://mlsanity.com/paper/2503.03562/) (2025)

## Abstract (arXiv)

Frame reconstruction (current or future frame) based on Auto-Encoder (AE) is a popular method for video anomaly detection. With models trained on the normal data, the reconstruction errors of anomalous scenes are usually much larger than those of normal ones. Previous methods introduced the memory bank into AE, for encoding diverse normal patterns across the training videos. However, they are memory-consuming and cannot cope with unseen new scenarios in the testing data. In this work, we propose a dynamic prototype unit (DPU) to encode the normal dynamics as prototypes in real time, free from extra memory cost. In addition, we introduce meta-learning to our DPU to form a novel few-shot normalcy learner, namely Meta-Prototype Unit (MPU). It enables the fast adaption capability on new scenes by only consuming a few iterations of update. Extensive experiments are conducted on various benchmarks. The superior performance over the state-of-the-art demonstrates the effectiveness of our method.