# Hypothesis Stitcher for End-to-End Speaker-attributed ASR on Long-form Multi-talker Recordings

arXiv: https://arxiv.org/abs/2101.01853 · HTML: https://mlsanity.com/paper/2101.01853/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 6 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [Serialized Output Training for End-to-End Overlapped Speech Recognition](https://mlsanity.com/paper/2003.12687/) (2020)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [Continuous speech separation: dataset and analysis](https://mlsanity.com/paper/2001.11482/) (2020)
- [Joint Speech Recognition and Speaker Diarization via Sequence Transduction](https://mlsanity.com/paper/1907.05337/) (2019)
- [A comparison of end-to-end models for long-form speech recognition](https://mlsanity.com/paper/1911.02242/) (2019)
- [Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakers](https://mlsanity.com/paper/2006.10930/) (2020)
- [Investigation of End-To-End Speaker-Attributed ASR for Continuous Multi-Talker Recordings](https://mlsanity.com/paper/2008.04546/) (2020)
- [RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions](https://mlsanity.com/paper/2005.03271/) (2020)
- [Advances in Online Audio-Visual Meeting Transcription](https://mlsanity.com/paper/1912.04979/) (2019)
- [Speech Recognition and Multi-Speaker Diarization of Long Conversations](https://mlsanity.com/paper/2005.08072/) (2020)
- [Simultaneous Speech Recognition and Speaker Diarization for Monaural Dialogue Recordings with Target-Speaker Acoustic Models](https://mlsanity.com/paper/1909.08103/) (2019)
- [Guided Source Separation Meets a Strong ASR Backend: Hitachi/Paderborn University Joint Investigation for Dinner Party ASR](https://mlsanity.com/paper/1905.12230/) (2019)

## Built on by (most cited first, 7 of 7)

- [Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone](https://mlsanity.com/paper/2103.16776/) (2021)
- [A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio](https://mlsanity.com/paper/2107.02852/) (2021)
- [Improving the Naturalness of Simulated Conversations for End-to-End Neural Diarization](https://mlsanity.com/paper/2204.11232/) (2022)
- [Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech](https://mlsanity.com/paper/2112.05863/) (2021)
- [Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio](https://mlsanity.com/paper/2505.10975/) (2025)
- [Improving Speaker Assignment in Speaker-Attributed ASR for Real Meeting Applications](https://mlsanity.com/paper/2403.06570/) (2024)
- [A Glance is Enough: Extract Target Sentence By Looking at A keyword](https://mlsanity.com/paper/2310.05352/) (2023)

## Abstract (arXiv)

An end-to-end (E2E) speaker-attributed automatic speech recognition (SA-ASR) model was proposed recently to jointly perform speaker counting, speech recognition and speaker identification. The model achieved a low speaker-attributed word error rate (SA-WER) for monaural overlapped speech comprising an unknown number of speakers. However, the E2E modeling approach is susceptible to the mismatch between the training and testing conditions. It has yet to be investigated whether the E2E SA-ASR model works well for recordings that are much longer than samples seen during training. In this work, we first apply a known decoding technique that was developed to perform single-speaker ASR for long-form audio to our E2E SA-ASR task. Then, we propose a novel method using a sequence-to-sequence model, called hypothesis stitcher. The model takes multiple hypotheses obtained from short audio segments that are extracted from the original long-form input, and it then outputs a fused single hypothesis. We propose several architectural variations of the hypothesis stitcher model and compare them with the conventional decoding methods. Experiments using LibriSpeech and LibriCSS corpora show that the proposed method significantly improves SA-WER especially for long-form multi-talker recordings.