mlsanityData to 7 Oct 2026

Probabilistic Fusion and Calibration of Neural Speaker Diarization Models

arXiv 2511.22696 ↗First posted 27 Nov 2025cs.SD cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

End-to-End Neural Diarization (EEND) systems produce frame-level probabilistic speaker activity estimates, yet since evaluation focuses primarily on Diarization Error Rate (DER), the reliability and calibration of these confidence scores have been largely neglected. When fusing multiple diarization systems, DOVER-Lap remains the only established approach, operating at the segment level with hard decisions. We propose working with continuous probability outputs, which enables more sophisticated fusion and calibration techniques that can leverage model uncertainty and complementary strengths across different architectures. This paper presents the first comprehensive framework for calibrating and fusing EEND models at the probability level. We investigate two output formulations (multilabel and powerset representations) and their impact on calibration and fusion effectiveness. Through extensive experiments on the CallHome two-speaker benchmark, we demonstrate that proper calibration provides substantial improvements even for individual models (up to 19% relative DER reduction), in some cases mitigating the absence of domain adaptation. We reveal that joint calibration in powerset space consistently outperforms independent per-speaker calibration, that fusion substantially improves over individual models, and that the Fuse-then-Calibrate ordering generally outperforms both calibrating before fusion and uncalibrated fusion while requiring calibration of only a single combined model. Our best configuration outperforms DOVER-Lap in terms of DER while providing reliable confidence estimates essential for downstream applications. This work proposes best practices for probability-level fusion of EEND systems and demonstrates the advantages of leveraging soft outputs over hard decisions.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 27 Nov 2025.

Builds on

22 of 39 references.

References · 22

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 400
  4. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  5. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 86
  6. 2019End-to-End Neural Speaker Diarization with Self-attentioncited by 74
  7. 2020End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractorscited by 70
  8. 2020Integrating end-to-end neural and clustering-based diarization: Getting the best of both worldscited by 33
  9. 2024Leveraging Self-Supervised Learning for Speaker Diarizationcited by 21
  10. 2022From Simulated Mixtures to Simulated Conversations as Training Data for End-to-End Neural Diarizationcited by 20
  11. 2020DOVER-Lap: A Method for Combining Overlap-aware Diarization Outputscited by 18
  12. 2021Overlap-aware low-latency online speaker diarization based on end-to-end local segmentationcited by 14
10 more
  1. 2024Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systemscited by 12
  2. 2024Evaluating Posterior Probabilities: Decision Theory, Proper Scoring Rules, and Calibrationcited by 11
  3. 2022EEND-SS: Joint End-to-End Neural Speaker Diarization and Speech Separation for Flexible Number of Speakerscited by 10
  4. 2022Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysiscited by 7
  5. 2019DOVER: A Method for Combining Diarization Outputscited by 6
  6. 2023Attention-based Encoder-Decoder End-to-End Neural Diarization with Embedding Enhancercited by 6
  7. 2023DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractorscited by 6
  8. 2022ASR-Aware End-to-end Neural Diarizationcited by 4
  9. 2023Attention-based Encoder-Decoder Network for End-to-End Neural Speaker Diarization with Target Speaker Attractorcited by 4
  10. 2024On the calibration of powerset speaker diarization modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.