# An Audio-Visual Dataset and Deep Learning Frameworks for Crowded Scene Classification

arXiv: https://arxiv.org/abs/2112.09172 · HTML: https://mlsanity.com/paper/2112.09172/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 16 Dec 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Learning from Between-class Examples for Deep Sound Recognition](https://mlsanity.com/paper/1711.10282/) (2017)
- [Robust Acoustic Scene Classification using a Multi-Spectrogram Encoder-Decoder Framework](https://mlsanity.com/paper/2002.04502/) (2020)
- [Multi-view Audio and Music Classification](https://mlsanity.com/paper/2103.02420/) (2021)
- [Deep Feature Embedding and Hierarchical Classification for Audio Scene Classification](https://mlsanity.com/paper/2002.04857/) (2020)
- [Sound Context Classification Basing on Join Learning Model and Multi-Spectrogram Features](https://mlsanity.com/paper/2005.12779/) (2020)
- [A Low-Compexity Deep Learning Framework For Acoustic Scene Classification](https://mlsanity.com/paper/2106.06838/) (2021)

## Built on by (most cited first, 5 of 5)

- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [Wider or Deeper Neural Network Architecture for Acoustic Scene Classification with Mismatched Recording Devices](https://mlsanity.com/paper/2203.12314/) (2022)
- [Remote Sensing Image Classification using Transfer Learning and Attention Based Deep Neural Network](https://mlsanity.com/paper/2206.13392/) (2022)
- [Robust, General, and Low Complexity Acoustic Scene Classification Systems and An Effective Visualization for Presenting a Sound Scene Context](https://mlsanity.com/paper/2210.08610/) (2022)
- [A Toolchain for Comprehensive Audio/Video Analysis Using Deep Learning Based Multimodal Approach (A use case of riot or violent context detection)](https://mlsanity.com/paper/2407.03110/) (2024)

## Abstract (arXiv)

This paper presents a task of audio-visual scene classification (SC) where input videos are classified into one of five real-life crowded scenes: 'Riot', 'Noise-Street', 'Firework-Event', 'Music-Event', and 'Sport-Atmosphere'. To this end, we firstly collect an audio-visual dataset (videos) of these five crowded contexts from Youtube (in-the-wild scenes). Then, a wide range of deep learning frameworks are proposed to deploy either audio or visual input data independently. Finally, results obtained from high-performed deep learning frameworks are fused to achieve the best accuracy score. Our experimental results indicate that audio and visual input factors independently contribute to the SC task's performance. Significantly, an ensemble of deep learning frameworks exploring either audio or visual input data can achieve the best accuracy of 95.7%.