# Toward Practical Automatic Speech Recognition and Post-Processing: a Call for Explainable Error Benchmark Guideline

arXiv: https://arxiv.org/abs/2401.14625 · HTML: https://mlsanity.com/paper/2401.14625/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 26 Jan 2024.

## Builds on (most cited first, 15 of 15)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Towards end-to-end spoken language understanding](https://mlsanity.com/paper/1802.08395/) (2018)
- [A spelling correction model for end-to-end speech recognition](https://mlsanity.com/paper/1902.07178/) (2019)
- [Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition](https://mlsanity.com/paper/2205.03433/) (2022)
- [Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model](https://mlsanity.com/paper/1910.10697/) (2019)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Improving Readability for Automatic Speech Recognition Transcription](https://mlsanity.com/paper/2004.04438/) (2020)
- [Data Augmentation for Training Dialog Models Robust to Speech Recognition Errors](https://mlsanity.com/paper/2006.05635/) (2020)
- [BembaSpeech: A Speech Recognition Corpus for the Bemba Language](https://mlsanity.com/paper/2102.04889/) (2021)
- [Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation](https://mlsanity.com/paper/2210.14389/) (2022)
- [CI-AVSR: A Cantonese Audio-Visual Speech Dataset for In-car Command Recognition](https://mlsanity.com/paper/2201.03804/) (2022)
- [UserLibri: A Dataset for ASR Personalization Using Only Text](https://mlsanity.com/paper/2207.00706/) (2022)
- [KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos](https://mlsanity.com/paper/1903.00216/) (2019)

## Built on by (most cited first, 4 of 4)

- [Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models](https://mlsanity.com/paper/2606.14647/) (2026)
- [Semantically Corrected Amharic Automatic Speech Recognition](https://mlsanity.com/paper/2404.13362/) (2024)
- [Exploring the Potential of Multimodal LLM with Knowledge-Intensive Multimodal ASR](https://mlsanity.com/paper/2406.10880/) (2024)
- [Framework for Curating Speech Datasets and Evaluating ASR Systems: A Case Study for Polish](https://mlsanity.com/paper/2408.00005/) (2024)

## Abstract (arXiv)

Automatic speech recognition (ASR) outcomes serve as input for downstream tasks, substantially impacting the satisfaction level of end-users. Hence, the diagnosis and enhancement of the vulnerabilities present in the ASR model bear significant importance. However, traditional evaluation methodologies of ASR systems generate a singular, composite quantitative metric, which fails to provide comprehensive insight into specific vulnerabilities. This lack of detail extends to the post-processing stage, resulting in further obfuscation of potential weaknesses. Despite an ASR model's ability to recognize utterances accurately, subpar readability can negatively affect user satisfaction, giving rise to a trade-off between recognition accuracy and user-friendliness. To effectively address this, it is imperative to consider both the speech-level, crucial for recognition accuracy, and the text-level, critical for user-friendliness. Consequently, we propose the development of an Error Explainable Benchmark (EEB) dataset. This dataset, while considering both speech- and text-level, enables a granular understanding of the model's shortcomings. Our proposition provides a structured pathway for a more `real-world-centric' evaluation, a marked shift away from abstracted, traditional methods, allowing for the detection and rectification of nuanced system weaknesses, ultimately aiming for an improved user experience.