# WASD: A Wilder Active Speaker Detection Dataset

arXiv: https://arxiv.org/abs/2303.05321 · HTML: https://mlsanity.com/paper/2303.05321/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [LRS3-TED: a large-scale dataset for visual speech recognition](https://mlsanity.com/paper/1809.00496/) (2018)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark](https://mlsanity.com/paper/1812.00324/) (2018)
- [Self-Supervised Learning of Audio-Visual Objects from Video](https://mlsanity.com/paper/2008.04237/) (2020)
- [Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection](https://mlsanity.com/paper/2107.06592/) (2021)
- [AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection](https://mlsanity.com/paper/1901.01342/) (2019)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [Spot the conversation: speaker diarisation in the wild](https://mlsanity.com/paper/2007.01216/) (2020)
- [Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning](https://mlsanity.com/paper/2008.05789/) (2020)
- [Perfect match: Improved cross-modal embeddings for audio-visual synchronisation](https://mlsanity.com/paper/1809.08001/) (2018)
- [Active Speakers in Context](https://mlsanity.com/paper/2005.09812/) (2020)
- [MAAS: Multi-modal Assignation for Active Speaker Detection](https://mlsanity.com/paper/2101.03682/) (2021)
- [How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild](https://mlsanity.com/paper/2106.03932/) (2021)
- [EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments](https://mlsanity.com/paper/2107.04174/) (2021)
- [Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection](https://mlsanity.com/paper/2207.07783/) (2022)
- [UniCon: Unified Context Network for Robust Active Speaker Detection](https://mlsanity.com/paper/2108.02607/) (2021)
- [Naver at ActivityNet Challenge 2019 -- Task B Active Speaker Detection (AVA)](https://mlsanity.com/paper/1906.10555/) (2019)
- [Personal VAD: Speaker-Conditioned Voice Activity Detection](https://mlsanity.com/paper/1908.04284/) (2019)
- [End-to-End Active Speaker Detection](https://mlsanity.com/paper/2203.14250/) (2022)
- [Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion](https://mlsanity.com/paper/1603.09725/) (2016)
- [Who said that?: Audio-visual speaker diarisation of real-world meetings](https://mlsanity.com/paper/1906.10042/) (2019)
- [Cross-modal Supervision for Learning Active Speaker Detection in Video](https://mlsanity.com/paper/1603.08907/) (2016)
- [Look Who's Talking: Active Speaker Detection in the Wild](https://mlsanity.com/paper/2108.07640/) (2021)

## Built on by (most cited first, 6 of 6)

- [ASDnB: Merging Face with Body Cues For Robust Active Speaker Detection](https://mlsanity.com/paper/2412.08594/) (2024)
- [GateFusion: Hierarchical Gated Cross-Modal Fusion for Active Speaker Detection](https://mlsanity.com/paper/2512.15707/) (2025)
- [FabuLight-ASD: Unveiling Speech Activity via Body Language](https://mlsanity.com/paper/2411.13674/) (2024)
- [C3ASD: Multi-Level Consistency-Driven Representation Learning for Robust Active Speaker Detection](https://mlsanity.com/paper/2607.03018/) (2026)
- [ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion](https://mlsanity.com/paper/2609.26648/) (2026)
- [CAST-ASD: Causal Attention over Speakers and Time for Online Active Speaker Detection](https://mlsanity.com/paper/2610.14676/) (2026)

## Abstract (arXiv)

Current Active Speaker Detection (ASD) models achieve great results on AVA-ActiveSpeaker (AVA), using only sound and facial features. Although this approach is applicable in movie setups (AVA), it is not suited for less constrained conditions. To demonstrate this limitation, we propose a Wilder Active Speaker Detection (WASD) dataset, with increased difficulty by targeting the two key components of current ASD: audio and face. Grouped into 5 categories, ranging from optimal conditions to surveillance settings, WASD contains incremental challenges for ASD with tactical impairment of audio and face data. We select state-of-the-art models and assess their performance in two groups of WASD: Easy (cooperative settings) and Hard (audio and/or face are specifically degraded). The results show that: 1) AVA trained models maintain a state-of-the-art performance in WASD Easy group, while underperforming in the Hard one, showing the 2) similarity between AVA and Easy data; and 3) training in WASD does not improve models performance to AVA levels, particularly for audio impairment and surveillance settings. This shows that AVA does not prepare models for wild ASD and current approaches are subpar to deal with such conditions. The proposed dataset also contains body data annotations to provide a new source for ASD, and is available at https://github.com/Tiago-Roxo/WASD.