# On Word Error Rate Definitions and their Efficient Computation for Multi-Speaker Speech Recognition Systems

arXiv: https://arxiv.org/abs/2211.16112 · HTML: https://mlsanity.com/paper/2211.16112/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2022.

## Built on by (most cited first, 5 of 5)

- [MeetEval: A Toolkit for Computation of Word Error Rates for Meeting Transcription Systems](https://mlsanity.com/paper/2307.11394/) (2023)
- [Combining TF-GridNet and Mixture Encoder for Continuous Speech Separation for Meeting Transcription](https://mlsanity.com/paper/2309.08454/) (2023)
- [Transcription-Free Fine-Tuning of Speech Separation Models for Noisy and Reverberant Multi-Speaker Automatic Speech Recognition](https://mlsanity.com/paper/2406.08914/) (2024)
- [Whispy: Adapting STT Whisper Models to Real-Time Environments](https://mlsanity.com/paper/2405.03484/) (2024)
- [Modeling Overlapped Speech with Shuffles](https://mlsanity.com/paper/2603.17769/) (2026)

## Abstract (arXiv)

We propose a general framework to compute the word error rate (WER) of ASR systems that process recordings containing multiple speakers at their input and that produce multiple output word sequences (MIMO). Such ASR systems are typically required, e.g., for meeting transcription. We provide an efficient implementation based on a dynamic programming search in a multi-dimensional Levenshtein distance tensor under the constraint that a reference utterance must be matched consistently with one hypothesis output. This also results in an efficient implementation of the ORC WER which previously suffered from exponential complexity. We give an overview of commonly used WER definitions for multi-speaker scenarios and show that they are specializations of the above MIMO WER tuned to particular application scenarios. We conclude with a discussion of the pros and cons of the various WER definitions and a recommendation when to use which.