mlsanityData to 7 Oct 2026

AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection

arXiv 1901.01342 ↗First posted 5 Jan 2019cs.CV cs.MM cs.SDcited by 81

Not on the map. Why?

Abstract (arXiv)

Active speaker detection is an important component in video analysis algorithms for applications such as speaker diarization, video re-targeting for meetings, speech enhancement, and human-robot interaction. The absence of a large, carefully labeled audio-visual dataset for this task has constrained algorithm evaluations with respect to data diversity, environments, and accuracy. This has made comparisons and improvements difficult. In this paper, we present the AVA Active Speaker detection dataset (AVA-ActiveSpeaker) that will be released publicly to facilitate algorithm development and enable comparisons. The dataset contains temporally labeled face tracks in video, where each face instance is labeled as speaking or not, and whether the speech is audible. This dataset contains about 3.65 million human labeled frames or about 38.5 hours of face tracks, and the corresponding audio. We also present a new audio-visual approach for active speaker detection, and analyze its performance, demonstrating both its strength and the contributions of the dataset.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 22; 2023: 14; 2024: 10; 2025: 10; 2026: 9.

81 papers.

Sort:

By area:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  3. 2021Sub-word Level Lip Reading With Visual Attentioncited by 60
  4. 2021ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learningcited by 50
  5. 2023RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatarscited by 45
  6. 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
  7. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  8. 2023A Light Weight Model for Active Speaker Detectioncited by 39
  9. 2021MAAS: Multi-modal Assignation for Active Speaker Detectioncited by 35
  10. 2021The Right to Talk: An Audio-Visual Transformer Approachcited by 33
  11. 2023Novel-View Acoustic Synthesiscited by 33
  12. 2021How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wildcited by 30
38 more
  1. 2022Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detectioncited by 29
  2. 2021UniCon: Unified Context Network for Robust Active Speaker Detectioncited by 27
  3. 2022Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localizationcited by 27
  4. 2021AVA-AVD: Audio-Visual Speaker Diarization in the Wildcited by 24
  5. 2022End-to-End Active Speaker Detectioncited by 24
  6. 2023LoCoNet: Long-Short Context Network for Active Speaker Detectioncited by 24
  7. 2021Look Who's Talking: Active Speaker Detection in the Wildcited by 21
  8. 2022Look&Listen: Multi-Modal Correlation Learning for Active Speaker Detection and Speech Enhancementcited by 20
  9. 2023Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipationcited by 20
  10. 2025See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Modelscited by 19
  11. 2023Egocentric Auditory Attention Localization in Conversationscited by 18
  12. 2021Structure from Silence: Learning Scene Structure from Ambient Soundcited by 17
  13. 2023AdVerb: Visually Guided Audio Dereverberationcited by 15
  14. 2023TalkNCE: Improving Active Speaker Detection with Talk-Aware Contrastive Learningcited by 15
  15. 2024Joint Audio-Visual Idling Vehicle Detection with Streamlined Input Dependenciescited by 14
  16. 2022Finding Fallen Objects Via Asynchronous Audio-Visual Integrationcited by 13
  17. 2022Rethinking Audio-visual Synchronization for Active Speaker Detectioncited by 9
  18. 2022Intel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarizationcited by 9
  19. 2022Using Active Speaker Faces for Diarization in TV showscited by 7
  20. 2023A multimodal dynamical variational autoencoder for audiovisual speech representation learningcited by 6
  21. 2025LASER: Lip Landmark Assisted Speaker Detection for Robustnesscited by 6
  22. 2023WASD: A Wilder Active Speaker Detection Datasetcited by 5
  23. 2024Spherical World-Locking for Audio-Visual Localization in Egocentric Videoscited by 5
  24. 2024ASDnB: Merging Face with Body Cues For Robust Active Speaker Detectioncited by 5
  25. 2025Revisiting Active Speaker Detection: An In-the-Wild Benchmark for Generalization and Robustnesscited by 5
  26. 2021FaVoA: Face-Voice Association Favours Ambiguous Speaker Detectioncited by 4
  27. 2021Learning Spatial-Temporal Graphs for Active Speaker Detectioncited by 4
  28. 2022Audio visual character profiles for detecting background characters in entertainment mediacited by 4
  29. 2022Audio-Visual Activity Guided Cross-Modal Identity Association for Active Speaker Detectioncited by 4
  30. 2023A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Datasetcited by 4
  31. 2025Multimodal Conversation Structure Understandingcited by 4
  32. 2025AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understandingcited by 4
  33. 2021APES: Audiovisual Person Search in Untrimmed Videocited by 3
  34. 2021Active Speaker Detection as a Multi-Objective Optimization with Uncertainty-based Multimodal Fusioncited by 3
  35. 2022Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detectioncited by 3
  36. 2022Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledgecited by 3
  37. 2023A Real-Time Active Speaker Detection System Integrating an Audio-Visual Signal with a Spatial Querying Mechanismcited by 3
  38. 2022UniCon+: ICTCAS-UCAS Submission to the AVA-ActiveSpeaker Task at ActivityNet Challenge 2022cited by 2

50 most cited of 81 All 81 (JSON)