# DiaScriber: A Speech LLM for Joint Diarization and Transcription in Multi-Speaker Scenarios

arXiv: https://arxiv.org/abs/2608.22796 · HTML: https://mlsanity.com/paper/2608.22796/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 24 Aug 2026.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 18 of 18)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [Qwen3-ASR Technical Report](https://mlsanity.com/paper/2601.21337/) (2026)
- [Qwen3-TTS Technical Report](https://mlsanity.com/paper/2601.15621/) (2026)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration](https://mlsanity.com/paper/2501.14350/) (2025)
- [Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset](https://mlsanity.com/paper/2203.16844/) (2022)
- [VIBEVOICE-ASR Technical Report](https://mlsanity.com/paper/2601.18184/) (2026)
- [Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods](https://mlsanity.com/paper/2509.13785/) (2025)
- [LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech](https://mlsanity.com/paper/2601.18220/) (2026)
- [Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio](https://mlsanity.com/paper/2505.10975/) (2025)
- [MOSS Transcribe Diarize Technical Report](https://mlsanity.com/paper/2601.01554/) (2026)
- [SoulX-Transcriber: A Robust End-to-End Framework for Multi-Speaker Speech Transcription](https://mlsanity.com/paper/2606.02400/) (2026)
- [Speaker-Reasoner: Scaling Interaction Turns and Reasoning Patterns for Timestamped Speaker-Attributed ASR](https://mlsanity.com/paper/2604.03074/) (2026)
- [DM-ASR: Diarization-aware Multi-speaker ASR with Large Language Models](https://mlsanity.com/paper/2604.22467/) (2026)
- [Joint Learning Global-Local Speaker Classification to Enhance End-to-End Speaker Diarization and Recognition](https://mlsanity.com/paper/2603.25377/) (2026)

## Abstract (arXiv)

Multi-speaker automatic speech recognition (MSASR) aims to jointly predict content transcriptions, speaker identities, and timestamps, thereby addressing the key question of "who spoke what and when" and holds substantial practical value in real-world multi-speaker scenarios. However, MSASR still encounters considerable challenges in the presence of fast turn transitions, overlapping speech, and complex, diverse multi-speaker scenarios. In this work, we propose DiaScriber, an end-to-end multi-speaker diarization and transcription model built on a speech large language model. We first construct diverse data pipelines to cover a wide variety of multi-speaker scenarios and their complexities, including validation and refinement, turn-transition and overlapping-speech simulation, and multimodal annotation. Furthermore, DiaScriber is developed based on the pretrained version of Qwen3.5-Omni through a three-stage training strategy involving continual pretraining, supervised fine-tuning, and reinforcement learning. Experiments show that DiaScriber achieves superior performance over comparison methods across extensive multi-speaker scenario test sets and demonstrates outstanding generalization ability in unseen multi-speaker scenarios.