# Multimodal Attention-Enhanced Feature Fusion-based Weekly Supervised Anomaly Violence Detection

arXiv: https://arxiv.org/abs/2409.11223 · HTML: https://mlsanity.com/paper/2409.11223/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 89 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm](https://mlsanity.com/paper/2110.05208/) (2021)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection](https://mlsanity.com/paper/1903.07256/) (2019)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)
- [Generative Cooperative Learning for Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2203.03962/) (2022)
- [Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2302.05160/) (2023)
- [Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection](https://mlsanity.com/paper/2003.06780/) (2020)
- [CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection](https://mlsanity.com/paper/2011.12077/) (2020)
- [CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2212.05136/) (2022)
- [Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2303.12369/) (2023)
- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [Localizing Anomalies from Weakly-Labeled Videos](https://mlsanity.com/paper/2008.08944/) (2020)
- [Motion-Aware Feature for Improved Video Anomaly Detection](https://mlsanity.com/paper/1907.10211/) (2019)
- [Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2212.04090/) (2022)
- [Weakly Supervised Video Anomaly Detection via Center-guided Discriminative Learning](https://mlsanity.com/paper/2104.07268/) (2021)
- [Learning Prompt-Enhanced Context Features for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2306.14451/) (2023)
- [Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detection](https://mlsanity.com/paper/2207.05500/) (2022)
- [VLTinT: Visual-Linguistic Transformer-in-Transformer for Coherent Video Paragraph Captioning](https://mlsanity.com/paper/2211.15103/) (2022)

## Built on by (most cited first, 2 of 2)

- [AVAR-Net: A Lightweight Audio-Visual Anomaly Recognition Framework with a Benchmark Dataset](https://mlsanity.com/paper/2510.13630/) (2025)
- [Benchmarking Compact VLMs for Clip-Level Surveillance Anomaly Detection Under Weak Supervision](https://mlsanity.com/paper/2603.13306/) (2026)

## Abstract (arXiv)

Weakly supervised video anomaly detection (WS-VAD) is a crucial area in computer vision for developing intelligent surveillance systems. This system uses three feature streams: RGB video, optical flow, and audio signals, where each stream extracts complementary spatial and temporal features using an enhanced attention module to improve detection accuracy and robustness. In the first stream, we employed an attention-based, multi-stage feature enhancement approach to improve spatial and temporal features from the RGB video where the first stage consists of a ViT-based CLIP module, with top-k features concatenated in parallel with I3D and Temporal Contextual Aggregation (TCA) based rich spatiotemporal features. The second stage effectively captures temporal dependencies using the Uncertainty-Regulated Dual Memory Units (UR-DMU) model, which learns representations of normal and abnormal data simultaneously, and the third stage is employed to select the most relevant spatiotemporal features. The second stream extracted enhanced attention-based spatiotemporal features from the flow data modality-based feature by taking advantage of the integration of the deep learning and attention module. The audio stream captures auditory cues using an attention module integrated with the VGGish model, aiming to detect anomalies based on sound patterns. These streams enrich the model by incorporating motion and audio signals often indicative of abnormal events undetectable through visual analysis alone. The concatenation of the multimodal fusion leverages the strengths of each modality, resulting in a comprehensive feature set that significantly improves anomaly detection accuracy and robustness across three datasets. The extensive experiment and high performance with the three benchmark datasets proved the effectiveness of the proposed system over the existing state-of-the-art system.