# Spot the conversation: speaker diarisation in the wild

arXiv: https://arxiv.org/abs/2007.01216 · HTML: https://mlsanity.com/paper/2007.01216/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 30 of 79)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency](https://mlsanity.com/paper/2101.03149/) (2021)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022)
- [A Light Weight Model for Active Speaker Detection](https://mlsanity.com/paper/2303.04439/) (2023)
- [MAAS: Multi-modal Assignation for Active Speaker Detection](https://mlsanity.com/paper/2101.03682/) (2021)
- [UniCon: Unified Context Network for Robust Active Speaker Detection](https://mlsanity.com/paper/2108.02607/) (2021)
- [Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localization](https://mlsanity.com/paper/2201.01928/) (2022)
- [AVA-AVD: Audio-Visual Speaker Diarization in the Wild](https://mlsanity.com/paper/2111.14448/) (2021)
- [LoCoNet: Long-Short Context Network for Active Speaker Detection](https://mlsanity.com/paper/2301.08237/) (2023)
- [Look Who's Talking: Active Speaker Detection in the Wild](https://mlsanity.com/paper/2108.07640/) (2021)
- [Egocentric Auditory Attention Localization in Conversations](https://mlsanity.com/paper/2303.16024/) (2023)
- [TalkNCE: Improving Active Speaker Detection with Talk-Aware Contrastive Learning](https://mlsanity.com/paper/2309.12306/) (2023)
- [Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems](https://mlsanity.com/paper/2409.06656/) (2024)
- [StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification](https://mlsanity.com/paper/2411.07076/) (2024)
- [Rethinking Audio-visual Synchronization for Active Speaker Detection](https://mlsanity.com/paper/2206.10421/) (2022)
- [VoxSRC 2022: The Fourth VoxCeleb Speaker Recognition Challenge](https://mlsanity.com/paper/2302.10248/) (2023)
- [The VoxCeleb Speaker Recognition Challenge: A Retrospective](https://mlsanity.com/paper/2408.14886/) (2024)
- [Web-Scale Collection of Video Data for 4D Animal Reconstruction](https://mlsanity.com/paper/2511.01169/) (2025)
- [Three-class Overlapped Speech Detection using a Convolutional Recurrent Neural Network](https://mlsanity.com/paper/2104.02878/) (2021)
- [Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling](https://mlsanity.com/paper/2401.12039/) (2024)
- [USTC-NELSLIP System Description for DIHARD-III Challenge](https://mlsanity.com/paper/2103.10661/) (2021)
- [Multi-scale speaker embedding-based graph attention networks for speaker diarisation](https://mlsanity.com/paper/2110.03361/) (2021)
- [Using Active Speaker Faces for Diarization in TV shows](https://mlsanity.com/paper/2203.15961/) (2022)
- [Target Speaker Voice Activity Detection with Transformers and Its Integration with End-to-End Neural Diarization](https://mlsanity.com/paper/2208.13085/) (2022)
- [Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization](https://mlsanity.com/paper/2302.12716/) (2023)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [WASD: A Wilder Active Speaker Detection Dataset](https://mlsanity.com/paper/2303.05321/) (2023)

## Abstract (arXiv)

The goal of this paper is speaker diarisation of videos collected 'in the wild'. We make three key contributions. First, we propose an automatic audio-visual diarisation method for YouTube videos. Our method consists of active speaker detection using audio-visual methods and speaker verification using self-enrolled speaker models. Second, we integrate our method into a semi-automatic dataset creation pipeline which significantly reduces the number of hours required to annotate videos with diarisation labels. Finally, we use this pipeline to create a large-scale diarisation dataset called VoxConverse, collected from 'in the wild' videos, which we will release publicly to the research community. Our dataset consists of overlapping speech, a large and diverse speaker pool, and challenging background conditions.