# Multi-level Memory-augmented Appearance-Motion Correspondence Framework for Video Anomaly Detection

arXiv: https://arxiv.org/abs/2303.05116 · HTML: https://mlsanity.com/paper/2303.05116/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction](https://mlsanity.com/paper/2108.06852/) (2021)
- [Anomaly Detection in Video Sequence with Appearance-Motion Correspondence](https://mlsanity.com/paper/1908.06351/) (2019)
- [Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm](https://mlsanity.com/paper/2004.07657/) (2020)
- [Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection](https://mlsanity.com/paper/1901.08954/) (2019)
- [Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2107.13118/) (2021)
- [Learning Normal Dynamics in Videos with Meta Prototype Network](https://mlsanity.com/paper/2104.06689/) (2021)
- [Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events](https://mlsanity.com/paper/2008.11988/) (2020)
- [Learning Appearance-motion Normality for Video Anomaly Detection](https://mlsanity.com/paper/2207.13361/) (2022)
- [Limiting the Reconstruction Capability of Generative Neural Network using Negative Learning](https://mlsanity.com/paper/1708.08985/) (2017)

## Built on by (most cited first, 5 of 5)

- [Evaluating the Effectiveness of Video Anomaly Detection in the Wild: Online Learning and Inference for Real-world Deployment](https://mlsanity.com/paper/2404.18747/) (2024)
- [A Survey on Video Anomaly Detection via Deep Learning: Human, Vehicle, and Environment](https://mlsanity.com/paper/2508.14203/) (2025)
- [ALFred: An Active Learning Framework for Real-world Semi-supervised Anomaly Detection with Adaptive Thresholds](https://mlsanity.com/paper/2508.09058/) (2025)
- [From Offline to Periodic Adaptation for Pose-Based Shoplifting Detection in Real-world Retail Security](https://mlsanity.com/paper/2603.04723/) (2026)
- [Motion Semantics Guided Normalizing Flow for Privacy-Preserving Video Anomaly Detection](https://mlsanity.com/paper/2603.26745/) (2026)

## Abstract (arXiv)

Frame prediction based on AutoEncoder plays a significant role in unsupervised video anomaly detection. Ideally, the models trained on the normal data could generate larger prediction errors of anomalies. However, the correlation between appearance and motion information is underutilized, which makes the models lack an understanding of normal patterns. Moreover, the models do not work well due to the uncontrollable generalizability of deep AutoEncoder. To tackle these problems, we propose a multi-level memory-augmented appearance-motion correspondence framework. The latent correspondence between appearance and motion is explored via appearance-motion semantics alignment and semantics replacement training. Besides, we also introduce a Memory-Guided Suppression Module, which utilizes the difference from normal prototype features to suppress the reconstruction capacity caused by skip-connection, achieving the tradeoff between the good reconstruction of normal data and the poor reconstruction of abnormal data. Experimental results show that our framework outperforms the state-of-the-art methods, achieving AUCs of 99.6%, 93.8%, and 76.3% on UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets.