# Audio-visual Representation Learning for Anomaly Events Detection in Crowds

arXiv: https://arxiv.org/abs/2110.14862 · HTML: https://mlsanity.com/paper/2110.14862/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 58 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [Learning from Synthetic Data for Crowd Counting in the Wild](https://mlsanity.com/paper/1903.03303/) (2019)
- [Ambient Sound Provides Supervision for Visual Learning](https://mlsanity.com/paper/1608.07017/) (2016)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning](https://mlsanity.com/paper/1712.07271/) (2017)
- [Self-supervised Moving Vehicle Tracking with Stereo Sound](https://mlsanity.com/paper/1910.11760/) (2019)
- [Unsupervised Deep Slow Feature Analysis for Change Detection in Multi-Temporal Remote Sensing Images](https://mlsanity.com/paper/1812.00645/) (2018)
- [A Self-Training Approach for Point-Supervised Object Detection and Counting in Crowds](https://mlsanity.com/paper/2007.12831/) (2020)
- [Reconstructive Sequence-Graph Network for Video Summarization](https://mlsanity.com/paper/2105.04066/) (2021)
- [PCC Net: Perspective Crowd Counting via Spatial Convolutional Network](https://mlsanity.com/paper/1905.10085/) (2019)
- [Learning Independent Instance Maps for Crowd Localization](https://mlsanity.com/paper/2012.04164/) (2020)
- [Pixel-wise Crowd Understanding via Synthetic Data](https://mlsanity.com/paper/2007.16032/) (2020)
- [Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification using ScatterNet Hybrid Deep Learning Network](https://mlsanity.com/paper/1806.00746/) (2018)
- [Curriculum Audiovisual Learning](https://mlsanity.com/paper/2001.09414/) (2020)
- [Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions](https://mlsanity.com/paper/2005.07097/) (2020)
- [Emotion-Based Crowd Representation for Abnormality Detection](https://mlsanity.com/paper/1607.07646/) (2016)

## Built on by (most cited first, 13 of 13)

- [Towards Efficient Partially Relevant Video Retrieval with Active Moment Discovering](https://mlsanity.com/paper/2504.10920/) (2025)
- [MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks](https://mlsanity.com/paper/2506.07016/) (2025)
- [EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception](https://mlsanity.com/paper/2506.21080/) (2025)
- [AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding](https://mlsanity.com/paper/2512.16250/) (2025)
- [Deep Crowd Anomaly Detection: State-of-the-Art, Challenges, and Future Research Directions](https://mlsanity.com/paper/2210.13927/) (2022)
- [Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectives](https://mlsanity.com/paper/2502.11858/) (2025)
- [A Survey on World Models Grounded in Acoustic Physical Information](https://mlsanity.com/paper/2506.13833/) (2025)
- [AVAR-Net: A Lightweight Audio-Visual Anomaly Recognition Framework with a Benchmark Dataset](https://mlsanity.com/paper/2510.13630/) (2025)
- [Imbalanced Aircraft Data Anomaly Detection](https://mlsanity.com/paper/2305.10082/) (2023)
- [Audio-Visual Dataset and Method for Anomaly Detection in Traffic Videos](https://mlsanity.com/paper/2305.15084/) (2023)
- [Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mapping](https://mlsanity.com/paper/2505.13777/) (2025)
- [Multi-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answering](https://mlsanity.com/paper/2511.23304/) (2025)
- [Synthetic-to-Real Translation for Class-Agnostic Motion Prediction](https://mlsanity.com/paper/2607.06319/) (2026)

## Abstract (arXiv)

In recent years, anomaly events detection in crowd scenes attracts many researchers' attention, because of its importance to public safety. Existing methods usually exploit visual information to analyze whether any abnormal events have occurred due to only visual sensors are generally equipped in public places. However, when an abnormal event in crowds occurs, sound information may be discriminative to assist the crowd analysis system to determine whether there is an abnormality. Compare with vision information that is easily occluded, audio signals have a certain degree of penetration. Thus, this paper attempt to exploit multi-modal learning for modeling the audio and visual signals simultaneously. To be specific, we design a two-branch network to model different types of information. The first is a typical 3D CNN model to extract temporal appearance features from video clips. The second is an audio CNN for encoding Log Mel-Spectrogram of audio signals. Finally, by fusing the above features, a more accurate prediction will be produced. We conduct the experiments on SHADE dataset, a synthetic audio-visual dataset in surveillance scenes, and find introducing audio signals effectively improves the performance of anomaly events detection and outperforms other state-of-the-art methods. Furthermore, we will release the code and the pre-trained models as soon as possible.