# Over-Tightening-Aware Pseudo-Labeling for Tight-Boundary Speaker Diarization

arXiv: https://arxiv.org/abs/2609.09965 · HTML: https://mlsanity.com/paper/2609.09965/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 50 of its 60 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 30 of 50)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [pyannote.audio: neural building blocks for speaker diarization](https://mlsanity.com/paper/1911.01255/) (2019)
- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [End-to-End Neural Speaker Diarization with Permutation-Free Objectives](https://mlsanity.com/paper/1909.05952/) (2019)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [End-to-End Neural Speaker Diarization with Self-attention](https://mlsanity.com/paper/1909.06247/) (2019)
- [Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario](https://mlsanity.com/paper/2005.07272/) (2020)
- [Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks](https://mlsanity.com/paper/2012.14952/) (2020)
- [Spot the conversation: speaker diarisation in the wild](https://mlsanity.com/paper/2007.01216/) (2020)
- [Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents](https://mlsanity.com/paper/2409.15594/) (2024)
- [Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds](https://mlsanity.com/paper/2010.13366/) (2020)
- [Learning Student-Friendly Teacher Networks for Knowledge Distillation](https://mlsanity.com/paper/2102.07650/) (2021)
- [Encoder-Decoder Based Attractors for End-to-End Neural Diarization](https://mlsanity.com/paper/2106.10654/) (2021)
- [MeetEval: A Toolkit for Computation of Word Error Rates for Meeting Transcription Systems](https://mlsanity.com/paper/2307.11394/) (2023)
- [Leveraging Self-Supervised Learning for Speaker Diarization](https://mlsanity.com/paper/2409.09408/) (2024)
- [Reshape Dimensions Network for Speaker Recognition](https://mlsanity.com/paper/2407.18223/) (2024)
- [GPU-accelerated Guided Source Separation for Meeting Transcription](https://mlsanity.com/paper/2212.05271/) (2022)
- [Nonlinear state-space identification using deep encoder networks](https://mlsanity.com/paper/2012.07697/) (2020)
- [TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings](https://mlsanity.com/paper/2303.03849/) (2023)
- [Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems](https://mlsanity.com/paper/2409.06656/) (2024)
- [Target Speaker Voice Activity Detection with Transformers and Its Integration with End-to-End Neural Diarization](https://mlsanity.com/paper/2208.13085/) (2022)

## Built on by (most cited first, 1 of 1)

- [Diarization Error Decomposition Under Pause Annotation Ambiguity](https://mlsanity.com/paper/2609.11007/) (2026)

## Abstract (arXiv)

Training speaker diarization models on loose labels, such as speech segments with padded boundaries or filled pauses, often results in similarly loose model outputs. To obtain tighter boundaries, pseudo-labeling based on the averaged outputs of causal and anticausal models has been proposed. However, since the pseudo-labels are estimation-based, they can suffer from over-tightening, which increases missed detections that can propagate as unrecoverable errors to downstream tasks. This paper carefully analyzes the causes of over-tightening and proposes three approaches to address them: (i) removing pause filling rather than padding, (ii) introducing a burn-in phase to mitigate missed detections near the beginning of causal and anticausal predictions, and (iii) making pseudo-label-based co-training aware of the non-causal model used for final inference. Experimental results show that the proposed method reduces missed detections caused by over-tightening and improves both diarization accuracy and downstream multi-talker ASR performance.