# Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks

arXiv: https://arxiv.org/abs/2012.14952 · HTML: https://mlsanity.com/paper/2012.14952/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2020.

## Built on by (most cited first, 30 of 73)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset](https://mlsanity.com/paper/2203.16844/) (2022)
- [AVA-AVD: Audio-Visual Speaker Diarization in the Wild](https://mlsanity.com/paper/2111.14448/) (2021)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communications](https://mlsanity.com/paper/2211.04054/) (2022)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems](https://mlsanity.com/paper/2409.06656/) (2024)
- [BERTraffic: BERT-based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communications](https://mlsanity.com/paper/2110.05781/) (2021)
- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [Multi-scale Speaker Diarization with Dynamic Scale Weighting](https://mlsanity.com/paper/2203.15974/) (2022)
- [Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond](https://mlsanity.com/paper/2502.03945/) (2025)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [Improving the Naturalness of Simulated Conversations for End-to-End Neural Diarization](https://mlsanity.com/paper/2204.11232/) (2022)
- [Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization](https://mlsanity.com/paper/2302.12716/) (2023)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [The Second DISPLACE Challenge : DIarization of SPeaker and LAnguage in Conversational Environments](https://mlsanity.com/paper/2406.09494/) (2024)
- [Deep Understanding of Sign Language for Sign to Subtitle Alignment](https://mlsanity.com/paper/2503.03287/) (2025)
- [In search of strong embedding extractors for speaker diarisation](https://mlsanity.com/paper/2210.14682/) (2022)
- [Summary of the DISPLACE Challenge 2023 - DIarization of SPeaker and LAnguage in Conversational Environments](https://mlsanity.com/paper/2311.12564/) (2023)
- [Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions](https://mlsanity.com/paper/2406.07890/) (2024)
- [Mamba-based Segmentation Model for Speaker Diarization](https://mlsanity.com/paper/2410.06459/) (2024)
- [Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio](https://mlsanity.com/paper/2505.10975/) (2025)
- [Multi-Channel End-to-End Neural Diarization with Distributed Microphones](https://mlsanity.com/paper/2110.04694/) (2021)
- [The Volcspeech system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge](https://mlsanity.com/paper/2202.04261/) (2022)
- [Magnitude-aware Probabilistic Speaker Embeddings](https://mlsanity.com/paper/2202.13826/) (2022)
- [The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines](https://mlsanity.com/paper/2208.08042/) (2022)

## Abstract (arXiv)

The recently proposed VBx diarization method uses a Bayesian hidden Markov model to find speaker clusters in a sequence of x-vectors. In this work we perform an extensive comparison of performance of the VBx diarization with other approaches in the literature and we show that VBx achieves superior performance on three of the most popular datasets for evaluating diarization: CALLHOME, AMI and DIHARDII datasets. Further, we present for the first time the derivation and update formulae for the VBx model, focusing on the efficiency and simplicity of this model as compared to the previous and more complex BHMM model working on frame-by-frame standard Cepstral features. Together with this publication, we release the recipe for training the x-vector extractors used in our experiments on both wide and narrowband data, and the VBx recipes that attain state-of-the-art performance on all three datasets. Besides, we point out the lack of a standardized evaluation protocol for AMI dataset and we propose a new protocol for both Beamformed and Mix-Headset audios based on the official AMI partitions and transcriptions.