# Toward Zero Oracle Word Error Rate on the Switchboard Benchmark

arXiv: https://arxiv.org/abs/2206.06192 · HTML: https://mlsanity.com/paper/2206.06192/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 13 Jun 2022.

## Builds on (most cited first, 7 of 7)

- [Achieving Human Parity in Conversational Speech Recognition](https://mlsanity.com/paper/1610.05256/) (2016)
- [Earnings-21: A Practical Benchmark for ASR in the Wild](https://mlsanity.com/paper/2104.11348/) (2021)
- [Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding](https://mlsanity.com/paper/2104.02138/) (2021)
- [English Conversational Telephone Speech Recognition by Humans and Machines](https://mlsanity.com/paper/1703.02136/) (2017)
- [On the limit of English conversational speech recognition](https://mlsanity.com/paper/2105.00982/) (2021)
- [Super-Human Performance in Online Low-latency Recognition of Conversational Speech](https://mlsanity.com/paper/2010.03449/) (2020)
- [Comparing Human and Machine Errors in Conversational Speech Transcription](https://mlsanity.com/paper/1708.08615/) (2017)

## Built on by (most cited first, 4 of 4)

- [SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluation](https://mlsanity.com/paper/2403.08196/) (2024)
- [Adapting Automatic Speech Recognition for Accented Air Traffic Control Communications](https://mlsanity.com/paper/2502.20311/) (2025)
- [Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing](https://mlsanity.com/paper/2607.18934/) (2026)
- [Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study](https://mlsanity.com/paper/2606.30237/) (2026)

## Abstract (arXiv)

The "Switchboard benchmark" is a very well-known test set in automatic speech recognition (ASR) research, establishing record-setting performance for systems that claim human-level transcription accuracy. This work highlights lesser-known practical considerations of this evaluation, demonstrating major improvements in word error rate (WER) by correcting the reference transcriptions and deviating from the official scoring methodology. In this more detailed and reproducible scheme, even commercial ASR systems can score below 5% WER and the established record for a research system is lowered to 2.3%. An alternative metric of transcript precision is proposed, which does not penalize deletions and appears to be more discriminating for human vs. machine performance. While commercial ASR systems are still below this threshold, a research system is shown to clearly surpass the accuracy of commercial human speech recognition. This work also explores using standardized scoring tools to compute oracle WER by selecting the best among a list of alternatives. A phrase alternatives representation is compared to utterance-level N-best lists and word-level data structures; using dense lattices and adding out-of-vocabulary words, this achieves an oracle WER of 0.18%.