# Improving Readability for Automatic Speech Recognition Transcription

arXiv: https://arxiv.org/abs/2004.04438 · HTML: https://mlsanity.com/paper/2004.04438/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2020.

## Built on by (most cited first, 22 of 22)

- [Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting](https://mlsanity.com/paper/2309.15649/) (2023)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition](https://mlsanity.com/paper/2109.14420/) (2021)
- [Neural Inverse Text Normalization](https://mlsanity.com/paper/2102.06380/) (2021)
- [Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction](https://mlsanity.com/paper/2211.13252/) (2022)
- [SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition](https://mlsanity.com/paper/2212.01039/) (2022)
- [Crossmodal ASR Error Correction with Discrete Speech Units](https://mlsanity.com/paper/2405.16677/) (2024)
- [RED-ACE: Robust Error Detection for ASR using Confidence Embeddings](https://mlsanity.com/paper/2203.07172/) (2022)
- [Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistants](https://mlsanity.com/paper/2405.19342/) (2024)
- [Four-in-One: A Joint Approach to Inverse Text Normalization, Punctuation, Capitalization, and Disfluency for Automatic Speech Recognition](https://mlsanity.com/paper/2210.15063/) (2022)
- [Toward Practical Automatic Speech Recognition and Post-Processing: a Call for Explainable Error Benchmark Guideline](https://mlsanity.com/paper/2401.14625/) (2024)
- [PATCorrect: Non-autoregressive Phoneme-augmented Transformer for ASR Error Correction](https://mlsanity.com/paper/2302.05040/) (2023)
- [Sentence-wise Speech Summarization: Task, Datasets, and End-to-End Modeling with LM Knowledge Distillation](https://mlsanity.com/paper/2408.00205/) (2024)
- [MathSpeech: Leveraging Small LMs for Accurate Conversion in Mathematical Speech-to-Formula](https://mlsanity.com/paper/2412.15655/) (2024)
- [Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model](https://mlsanity.com/paper/2102.11114/) (2021)
- [Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition](https://mlsanity.com/paper/2407.12817/) (2024)
- [End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data](https://mlsanity.com/paper/2311.17741/) (2023)
- [AgenticASR: Refining Speech Recognition in Real-World Scenarios via an Agentic Approach](https://mlsanity.com/paper/2607.28175/) (2026)
- [Zero-Shot Joint Modeling of Multiple Spoken-Text-Style Conversion Tasks using Switching Tokens](https://mlsanity.com/paper/2106.12131/) (2021)
- [Improving Audio Caption Fluency with Automatic Error Correction](https://mlsanity.com/paper/2306.10090/) (2023)
- [Boosting the Transferability of Audio Adversarial Examples with Acoustic Representation Optimization](https://mlsanity.com/paper/2503.19591/) (2025)
- [ASR Error Correction in Low-Resource Burmese with Alignment-Enhanced Transformers using Phonetic Features](https://mlsanity.com/paper/2511.21088/) (2025)

## Abstract (arXiv)

Modern Automatic Speech Recognition (ASR) systems can achieve high performance in terms of recognition accuracy. However, a perfectly accurate transcript still can be challenging to read due to grammatical errors, disfluency, and other errata common in spoken communication. Many downstream tasks and human readers rely on the output of the ASR system; therefore, errors introduced by the speaker and ASR system alike will be propagated to the next task in the pipeline. In this work, we propose a novel NLP task called ASR post-processing for readability (APR) that aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of the speaker. In addition, we describe a method to address the lack of task-specific data by synthesizing examples for the APR task using the datasets collected for Grammatical Error Correction (GEC) followed by text-to-speech (TTS) and ASR. Furthermore, we propose metrics borrowed from similar tasks to evaluate performance on the APR task. We compare fine-tuned models based on several open-sourced and adapted pre-trained models with the traditional pipeline method. Our results suggest that finetuned models improve the performance on the APR task significantly, hinting at the potential benefits of using APR systems. We hope that the read, understand, and rewrite approach of our work can serve as a basis that many NLP tasks and human readers can benefit from.