# Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions

arXiv: https://arxiv.org/abs/2005.07097 · HTML: https://mlsanity.com/paper/2005.07097/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2020.

## Built on by (most cited first, 13 of 13)

- [Few-shot Object Counting and Detection](https://mlsanity.com/paper/2207.10988/) (2022)
- [Perceptual Score: What Data Modalities Does Your Model Perceive?](https://mlsanity.com/paper/2110.14375/) (2021)
- [Multi-Exit Vision Transformer for Dynamic Inference](https://mlsanity.com/paper/2106.15183/) (2021)
- [Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead](https://mlsanity.com/paper/2105.09121/) (2021)
- [Self-supervised Audiovisual Representation Learning for Remote Sensing Data](https://mlsanity.com/paper/2108.00688/) (2021)
- [Audio-visual Representation Learning for Anomaly Events Detection in Crowds](https://mlsanity.com/paper/2110.14862/) (2021)
- [Audio-Visual Transformer Based Crowd Counting](https://mlsanity.com/paper/2109.01926/) (2021)
- [Efficient High-Resolution Deep Learning: A Survey](https://mlsanity.com/paper/2207.13050/) (2022)
- [Crowd Counting on Heavily Compressed Images with Curriculum Pre-Training](https://mlsanity.com/paper/2208.07075/) (2022)
- [Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight](https://mlsanity.com/paper/2212.02053/) (2022)
- [Repetitive Activity Counting by Sight and Sound](https://mlsanity.com/paper/2103.13096/) (2021)
- [MAFNet: A Multi-Attention Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2208.06761/) (2022)
- [Audio-Visual Dataset and Method for Anomaly Detection in Traffic Videos](https://mlsanity.com/paper/2305.15084/) (2023)

## Abstract (arXiv)

Visual crowd counting has been recently studied as a way to enable people counting in crowd scenes from images. Albeit successful, vision-based crowd counting approaches could fail to capture informative features in extreme conditions, e.g., imaging at night and occlusion. In this work, we introduce a novel task of audiovisual crowd counting, in which visual and auditory information are integrated for counting purposes. We collect a large-scale benchmark, named auDiovISual Crowd cOunting (DISCO) dataset, consisting of 1,935 images and the corresponding audio clips, and 170,270 annotated instances. In order to fuse the two modalities, we make use of a linear feature-wise fusion module that carries out an affine transformation on visual and auditory features. Finally, we conduct extensive experiments using the proposed dataset and approach. Experimental results show that introducing auditory information can benefit crowd counting under different illumination, noise, and occlusion conditions. The dataset and code will be released. Code and data have been made available