mlsanityData to 7 Oct 2026

CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection

arXiv 2212.05136 ↗First posted 9 Dec 2022cs.CVcited by 67

Not on the map. Why?

Abstract (arXiv)

Video anomaly detection (VAD) -- commonly formulated as a multiple-instance learning problem in a weakly-supervised manner due to its labor-intensive nature -- is a challenging problem in video surveillance where the frames of anomaly need to be localized in an untrimmed video. In this paper, we first propose to utilize the ViT-encoded visual features from CLIP, in contrast with the conventional C3D or I3D features in the domain, to efficiently extract discriminative representations in the novel technique. We then model temporal dependencies and nominate the snippets of interest by leveraging our proposed Temporal Self-Attention (TSA). The ablation study confirms the effectiveness of TSA and ViT feature. The extensive experiments show that our proposed CLIP-TSA outperforms the existing state-of-the-art (SOTA) methods by a large margin on three commonly-used benchmark datasets in the VAD problem (UCF-Crime, ShanghaiTech Campus, and XD-Violence). Our source code is available at https://github.com/joos2010kj/CLIP-TSA.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 73 of its 108 references that are in the map. First posted on arXiv 9 Dec 2022.

Builds on

73 of 108 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2017Non-local Neural Networkscited by 2,290
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
38 more
  1. 2018SlowFast Networks for Video Recognitioncited by 1,862
  2. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  3. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  5. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  7. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  8. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  9. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  10. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  11. 2017Dense-Captioning Events in Videoscited by 789
  12. 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
  13. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 560
  14. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  15. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  16. 2020Multi-modal Transformer for Video Retrievalcited by 395
  17. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  18. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  19. 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
  20. 2016Learning Temporal Regularity in Video Sequencescited by 254
  21. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 248
  22. 2020UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learningcited by 239
  23. 2020Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervisioncited by 235
  24. 2021Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learningcited by 226
  25. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  26. 2015A Dataset for Movie Descriptioncited by 195
  27. 2019Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detectioncited by 171
  28. 2020Human Action Recognition from Various Data Modalities: A Reviewcited by 162
  29. 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
  30. 2018Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Videocited by 110
  31. 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
  32. 2022Generative Cooperative Learning for Unsupervised Video Anomaly Detectioncited by 102
  33. 2021Contextual Transformer Networks for Visual Recognitioncited by 90
  34. 2018Localizing Moments in Video with Temporal Languagecited by 83
  35. 2020MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioningcited by 79
  36. 2020CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detectioncited by 79
  37. 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
  38. 2020Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Eventscited by 70

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 17; 2025: 25; 2026: 21.

67 papers.

Sort:

By area:

  1. 2023VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detectioncited by 121
  2. 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 85
  3. 2024Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularitycited by 66
  4. 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 63
  5. 2023Open-Vocabulary Video Anomaly Detectioncited by 57
  6. 2024Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detectioncited by 49
  7. 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
  8. 2024VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Modelscited by 24
  9. 2024Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detectioncited by 22
  10. 2024Video Anomaly Detection in 10 Years: A Survey and Outlookcited by 18
  11. 2025VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Treecited by 15
  12. 2025Anomize: Better Open Vocabulary Video Anomaly Detectioncited by 13
38 more
  1. 2023ZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detectioncited by 12
  2. 2025Flashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detectioncited by 12
  3. 2025MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detectioncited by 10
  4. 2024Exploring What Why and How: A Multifaceted Benchmark for Causation Understanding of Video Anomalycited by 9
  5. 2025Towards Visual Discrimination and Reasoning of Real-World Physical Dynamics: Physics-Grounded Anomaly Detectioncited by 9
  6. 2025A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysiscited by 9
  7. 2025Language-guided Open-world Video Anomaly Detection under Weak Supervisioncited by 8
  8. 2025Foundation Models and Transformers for Anomaly Detection: A Surveycited by 8
  9. 2025HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMscited by 8
  10. 2025Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Promptingcited by 8
  11. 2024Anomaly Detection for People with Visual Impairments Using an Egocentric 360-Degree Cameracited by 7
  12. 2025Aligning Effective Tokens with Video Anomaly in Large Language Modelscited by 7
  13. 2024Cross-Domain Learning for Video Anomaly Detection with Limited Supervisioncited by 6
  14. 2025AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detectioncited by 6
  15. 2025VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understandingcited by 6
  16. 2025Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detectioncited by 6
  17. 2026No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detectioncited by 6
  18. 2025Pistachio: Towards Synthetic, Balanced, and Long-Form Video Anomaly Benchmarkscited by 5
  19. 2025The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLMcited by 4
  20. 2025Rethinking Metrics and Benchmarks of Video Anomaly Detectioncited by 3
  21. 2025Learning Event Completeness for Weakly Supervised Video Anomaly Detectioncited by 3
  22. 2026Weakly Supervised Video Anomaly Detection with Anomaly-Connected Components and Intention Reasoningcited by 3
  23. 2024Learn Suspected Anomalies from Event Prompts for Video Anomaly Detectioncited by 2
  24. 2024Multimodal Attention-Enhanced Feature Fusion-based Weekly Supervised Anomaly Violence Detectioncited by 2
  25. 2024A Lightweight Dual-Branch System for Weakly-Supervised Video Anomaly Detection on Consumer Edge Devicescited by 2
  26. 2025FrameShield: Adversarially Robust Video Anomaly Detectioncited by 2
  27. 2026Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detectioncited by 2
  28. 2026SphereVAD: Training-Free Video Anomaly Detection via Geodesic Inference on the Unit Hyperspherecited by 2
  29. 2026MemoVAD: Resource-Efficient Video Anomaly Detection via Dynamic Semantic Memory in Edge Computing Scenarioscited by 2
  30. 2026Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindnesscited by 2
  31. 2023SolarFormer: Multi-scale Transformer for Solar PV Profilingcited by 1
  32. 2025Just Dance with π! A Poly-modal Inductor for Weakly-supervised Video Anomaly Detectioncited by 1
  33. 2025GV-VAD : Exploring Video Generation for Weakly-Supervised Video Anomaly Detectioncited by 1
  34. 2025Context-Aware Zero-Shot Anomaly Detection in Surveillance Using Contrastive and Predictive Spatiotemporal Modelingcited by 1
  35. 2025GTA-Crime: A Synthetic Dataset and Generation Framework for Fatal Violence Detection with Adversarial Snippet-Level Domain Adaptationcited by 1
  36. 2025TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detectioncited by 1
  37. 2026Towards Video Anomaly Detection from Event Streams: A Baseline and Benchmark Datasetscited by 1
  38. 2026COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detectioncited by 1

50 most cited of 67 All 67 (JSON)