# Achieving Human Parity in Conversational Speech Recognition

arXiv: https://arxiv.org/abs/1610.05256 · HTML: https://mlsanity.com/paper/1610.05256/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2016.

## Built on by (most cited first, 30 of 63)

- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data](https://mlsanity.com/paper/2101.07597/) (2021)
- [Robust Self-Supervised Audio-Visual Speech Recognition](https://mlsanity.com/paper/2201.01763/) (2022)
- [Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation](https://mlsanity.com/paper/2202.02628/) (2022)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories](https://mlsanity.com/paper/2111.04625/) (2021)
- [QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus](https://mlsanity.com/paper/2106.13000/) (2021)
- [A Survey of Multilingual Models for Automatic Speech Recognition](https://mlsanity.com/paper/2202.12576/) (2022)
- [The 2025 PNPL Competition: Speech Detection and Phoneme Classification in the LibriBrain Dataset](https://mlsanity.com/paper/2506.10165/) (2025)
- [Transformer Language Models with LSTM-based Cross-utterance Information Representation](https://mlsanity.com/paper/2102.06474/) (2021)
- [Online Continual Learning of End-to-End Speech Recognition Models](https://mlsanity.com/paper/2207.05071/) (2022)
- [Arabic Speech Recognition by End-to-End, Modular Systems and Human](https://mlsanity.com/paper/2101.08454/) (2021)
- [Accented Speech Recognition: A Survey](https://mlsanity.com/paper/2104.10747/) (2021)
- [Lethal Dose Conjecture on Data Poisoning](https://mlsanity.com/paper/2208.03309/) (2022)
- [Beating Backdoor Attack at Its Own Game](https://mlsanity.com/paper/2307.15539/) (2023)
- [Dompteur: Taming Audio Adversarial Examples](https://mlsanity.com/paper/2102.05431/) (2021)
- [The History of Speech Recognition to the Year 2030](https://mlsanity.com/paper/2108.00084/) (2021)
- [MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition](https://mlsanity.com/paper/2401.03424/) (2024)
- [Cross-utterance Reranking Models with BERT and Graph Convolutional Networks for Conversational Speech Recognition](https://mlsanity.com/paper/2106.06922/) (2021)
- [Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech](https://mlsanity.com/paper/2112.05863/) (2021)
- [Multilingual Word Error Rate Estimation: e-WER3](https://mlsanity.com/paper/2304.00649/) (2023)
- [Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intents](https://mlsanity.com/paper/2406.11047/) (2024)
- [Survey on the Convergence of Machine Learning and Blockchain](https://mlsanity.com/paper/2201.00976/) (2022)
- [Toward Zero Oracle Word Error Rate on the Switchboard Benchmark](https://mlsanity.com/paper/2206.06192/) (2022)
- [Enhancing Lip Reading with Multi-Scale Video and Multi-Encoder](https://mlsanity.com/paper/2404.05466/) (2024)
- [Focus Agent: LLM-Powered Virtual Focus Group](https://mlsanity.com/paper/2409.01907/) (2024)
- [Dialectal Speech Recognition and Translation of Swiss German Speech to Standard German Text: Microsoft's Submission to SwissText 2021](https://mlsanity.com/paper/2106.08126/) (2021)
- [User-Initiated Repetition-Based Recovery in Multi-Utterance Dialogue Systems](https://mlsanity.com/paper/2108.01208/) (2021)
- [Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet](https://mlsanity.com/paper/2406.17825/) (2024)

## Abstract (arXiv)

Conversational speech recognition has served as a flagship speech recognition task since the release of the Switchboard corpus in the 1990s. In this paper, we measure the human error rate on the widely used NIST 2000 test set, and find that our latest automated system has reached human parity. The error rate of professional transcribers is 5.9% for the Switchboard portion of the data, in which newly acquainted pairs of people discuss an assigned topic, and 11.3% for the CallHome portion where friends and family members have open-ended conversations. In both cases, our automated system establishes a new state of the art, and edges past the human benchmark, achieving error rates of 5.8% and 11.0%, respectively. The key to our system's performance is the use of various convolutional and LSTM acoustic model architectures, combined with a novel spatial smoothing method and lattice-free MMI acoustic training, multiple recurrent neural network language modeling approaches, and a systematic use of system combination.