# Leveraging Audio Representations for Vibration-Based Crowd Monitoring in Stadiums

arXiv: https://arxiv.org/abs/2503.17646 · HTML: https://mlsanity.com/paper/2503.17646/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [MRCNet: Crowd Counting and Density Map Estimation in Aerial and Ground Imagery](https://mlsanity.com/paper/1909.12743/) (2019)
- [High-quality Speech Synthesis Using Super-resolution Mel-Spectrogram](https://mlsanity.com/paper/1912.01167/) (2019)
- [HierMUD: Hierarchical Multi-task Unsupervised Domain Adaptation between Bridges for Drive-by Damage Diagnosis](https://mlsanity.com/paper/2107.11435/) (2021)
- [SMILE: Sequence-to-Sequence Domain Adaption with Minimizing Latent Entropy for Text Image Recognition](https://mlsanity.com/paper/2202.11949/) (2022)
- [WeVibe: Weight Change Estimation Through Audio-Induced Shelf Vibrations In Autonomous Stores](https://mlsanity.com/paper/2502.12093/) (2025)
- [Continual Person Identification using Footstep-Induced Floor Vibrations on Heterogeneous Floor Structures](https://mlsanity.com/paper/2502.15632/) (2025)

## Abstract (arXiv)

Crowd monitoring in sports stadiums is important to enhance public safety and improve the audience experience. Existing approaches mainly rely on cameras and microphones, which can cause significant disturbances and often raise privacy concerns. In this paper, we sense floor vibration, which provides a less disruptive and more non-intrusive way of crowd sensing, to predict crowd behavior. However, since the vibration-based crowd monitoring approach is newly developed, one main challenge is the lack of training data due to sports stadiums being large public spaces with complex physical activities. In this paper, we present ViLA (Vibration Leverage Audio), a vibration-based method that reduces the dependency on labeled data by pre-training with unlabeled cross-modality data. ViLA is first pre-trained on audio data in an unsupervised manner and then fine-tuned with a minimal amount of in-domain vibration data. By leveraging publicly available audio datasets, ViLA learns the wave behaviors from audio and then adapts the representation to vibration, reducing the reliance on domain-specific vibration data. Our real-world experiments demonstrate that pre-training the vibration model using publicly available audio data (YouTube8M) achieved up to a 5.8x error reduction compared to the model without audio pre-training.