# Quantifying the Role of Textual Predictability in Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2407.16537 · HTML: https://mlsanity.com/paper/2407.16537/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 23 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 3 of 3)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)

## Built on by (most cited first, 2 of 2)

- [FlanEC: Exploring Flan-T5 for Post-ASR Error Correction](https://mlsanity.com/paper/2501.12979/) (2025)
- [Investigating Transcription Normalization in the Faetar ASR Benchmark](https://mlsanity.com/paper/2508.11771/) (2025)

## Abstract (arXiv)

A long-standing question in automatic speech recognition research is how to attribute errors to the ability of a model to model the acoustics, versus its ability to leverage higher-order context (lexicon, morphology, syntax, semantics). We validate a novel approach which models error rates as a function of relative textual predictability, and yields a single number, k, which measures the effect of textual predictability on the recognizer. We use this method to demonstrate that a Wav2Vec 2.0-based model makes greater stronger use of textual context than a hybrid ASR model, in spite of not using an explicit language model, and also use it to shed light on recent results demonstrating poor performance of standard ASR systems on African-American English. We demonstrate that these mostly represent failures of acoustic--phonetic modelling. We show how this approach can be used straightforwardly in diagnosing and improving ASR.