mlsanityData to 7 Oct 2026

Video Abnormal Event Detection by Learning to Complete Visual Cloze Tests

arXiv 2108.02356 ↗First posted 5 Aug 2021cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Although deep neural networks (DNNs) enable great progress in video abnormal event detection (VAD), existing solutions typically suffer from two issues: (1) The localization of video events cannot be both precious and comprehensive. (2) The semantics and temporal context are under-explored. To tackle those issues, we are motivated by the prevalent cloze test in education and propose a novel approach named Visual Cloze Completion (VCC), which conducts VAD by learning to complete "visual cloze tests" (VCTs). Specifically, VCC first localizes each video event and encloses it into a spatio-temporal cube (STC). To achieve both precise and comprehensive localization, appearance and motion are used as complementary cues to mark the object region associated with each event. For each marked region, a normalized patch sequence is extracted from current and adjacent frames and stacked into a STC. With each patch and the patch sequence of a STC compared to a visual "word" and "sentence" respectively, we deliberately erase a certain "word" (patch) to yield a VCT. Then, the VCT is completed by training DNNs to infer the erased patch and its optical flow via video semantics. Meanwhile, VCC fully exploits temporal context by alternatively erasing each patch in temporal context and creating multiple VCTs. Furthermore, we propose localization-level, event-level, model-level and decision-level solutions to enhance VCC, which can further exploit VCC's potential and produce significant performance improvement gain. Extensive experiments demonstrate that VCC achieves state-of-the-art VAD performance. Our codes and results are open at https://github.com/yuguangnudt/VEC_VAD/tree/VCC.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 78 references that are in the map. First posted on arXiv 5 Aug 2021.

Builds on

35 of 78 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  6. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  7. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  8. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 560
  9. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  10. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  11. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  12. 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
23 more
  1. 2016Learning Temporal Regularity in Video Sequencescited by 254
  2. 2018Adversarially Learned One-Class Classifier for Novelty Detectioncited by 202
  3. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  4. 2019Anomaly Detection in Video Sequence with Appearance-Motion Correspondencecited by 118
  5. 2020Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigmcited by 115
  6. 2018Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Videocited by 110
  7. 2019Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videoscited by 103
  8. 2019Graph Embedded Pose Clustering for Anomaly Detectioncited by 95
  9. 2020Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detectioncited by 83
  10. 2017Abnormal Event Detection in Videos using Generative Adversarial Netscited by 82
  11. 2015Learning Deep Representations of Appearance and Motion for Anomalous Event Detectioncited by 75
  12. 2020Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Eventscited by 70
  13. 2013One-Class Classification: Taxonomy of Study and Review of Techniquescited by 68
  14. 2019Multi-timescale Trajectory Prediction for Abnormal Human Activity Detectioncited by 59
  15. 2017Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledgecited by 55
  16. 2016Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenescited by 50
  17. 2017Unmasking the abnormal events in videocited by 48
  18. 2020Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Predictioncited by 42
  19. 2016Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detectioncited by 32
  20. 2018Detecting abnormal events in video using Narrowed Normality Clusterscited by 30
  21. 2019Future Frame Prediction Using Convolutional VRNN for Anomaly Detectioncited by 27
  22. 2016A Discriminative Framework for Anomaly Detection in Large Videoscited by 25
  23. 2020Online Anomaly Detection in Surveillance Videos with Asymptotic Bounds on False Alarm Ratecited by 22

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Deep Crowd Anomaly Detection: State-of-the-Art, Challenges, and Future Research Directionscited by 3