# Explanations for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2302.14062 · HTML: https://mlsanity.com/paper/2302.14062/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 27 Feb 2023.

## Builds on (most cited first, 12 of 12)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Post-hoc Interpretability for Neural NLP: A Survey](https://mlsanity.com/paper/2108.04840/) (2021)
- [DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems](https://mlsanity.com/paper/1906.10263/) (2019)
- [Causes and Explanations: A Structural-Model Approach --- Part 1: Causes](https://mlsanity.com/paper/1301.2275/) (2013)
- [Confidence Estimation for Black Box Automatic Speech Recognition Systems Using Lattice Recurrent Neural Networks](https://mlsanity.com/paper/1910.11933/) (2019)
- [Explaining Image Classifiers using Statistical Fault Localization](https://mlsanity.com/paper/1908.02374/) (2019)

## Built on by (most cited first, 8 of 8)

- [Explainable Attribute-Based Speaker Verification](https://mlsanity.com/paper/2405.19796/) (2024)
- [Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognition](https://mlsanity.com/paper/2305.18011/) (2023)
- [Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features](https://mlsanity.com/paper/2309.07733/) (2023)
- [SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation](https://mlsanity.com/paper/2411.01710/) (2024)
- [On the reliability of feature attribution methods for speech classification](https://mlsanity.com/paper/2505.16406/) (2025)
- [Cross-Attention is Half Explanation in Speech-to-Text Models](https://mlsanity.com/paper/2509.18010/) (2025)
- [Towards Dys-XAI: Influence-Based Explanations for Dysarthria Severity Assessment](https://mlsanity.com/paper/2606.21306/) (2026)
- [Mask-Induced Displacement in Audio XAI via Logit Trajectory Decomposition](https://mlsanity.com/paper/2609.33486/) (2026)

## Abstract (arXiv)

We address quality assessment for neural network based ASR by providing explanations that help increase our understanding of the system and ultimately help build trust in the system. Compared to simple classification labels, explaining transcriptions is more challenging as judging their correctness is not straightforward and transcriptions as a variable-length sequence is not handled by existing interpretable machine learning models. We provide an explanation for an ASR transcription as a subset of audio frames that is both a minimal and sufficient cause of the transcription. To do this, we adapt existing explainable AI (XAI) techniques from image classification-Statistical Fault Localisation(SFL) and Causal. Additionally, we use an adapted version of Local Interpretable Model-Agnostic Explanations (LIME) for ASR as a baseline in our experiments. We evaluate the quality of the explanations generated by the proposed techniques over three different ASR ,Google API, the baseline model of Sphinx, Deepspeech and 100 audio samples from the Commonvoice dataset.