# DNCASR: End-to-End Training for Speaker-Attributed ASR

arXiv: https://arxiv.org/abs/2506.01916 · HTML: https://mlsanity.com/paper/2506.01916/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2025.

## Built on by (most cited first, 5 of 5)

- [D-ORCA: Dialogue-Centric Optimization for Robust Audio-Visual Captioning](https://mlsanity.com/paper/2602.07960/) (2026)
- [TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding](https://mlsanity.com/paper/2601.06896/) (2026)
- [Adapting Diarization-Conditioned Whisper for End-to-End Multi-Talker Speech Recognition](https://mlsanity.com/paper/2510.03723/) (2025)
- [Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics](https://mlsanity.com/paper/2603.22709/) (2026)
- [Cripping AI: Reimagining AI Through Lived Disability Experiences](https://mlsanity.com/paper/2605.02080/) (2026)

## Abstract (arXiv)

This paper introduces DNCASR, a novel end-to-end trainable system designed for joint neural speaker clustering and automatic speech recognition (ASR), enabling speaker-attributed transcription of long multi-party meetings. DNCASR uses two separate encoders to independently encode global speaker characteristics and local waveform information, along with two linked decoders to generate speaker-attributed transcriptions. The use of linked decoders allows the entire system to be jointly trained under a unified loss function. By employing a serialised training approach, DNCASR effectively addresses overlapping speech in real-world meetings, where the link improves the prediction of speaker indices in overlapping segments. Experiments on the AMI-MDM meeting corpus demonstrate that the jointly trained DNCASR outperforms a parallel system that does not have links between the speaker and ASR decoders. Using cpWER to measure the speaker-attributed word error rate, DNCASR achieves a 9.0% relative reduction on the AMI-MDM Eval set.