# Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognition

arXiv: https://arxiv.org/abs/2305.18011 · HTML: https://mlsanity.com/paper/2305.18011/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 26 of 26)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [Post-hoc Interpretability for Neural NLP: A Survey](https://mlsanity.com/paper/2108.04840/) (2021)
- [Debugging Tests for Model Explanations](https://mlsanity.com/paper/2011.05429/) (2020)
- [Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI](https://mlsanity.com/paper/2003.07258/) (2020)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data](https://mlsanity.com/paper/1808.02610/) (2018)
- [Do Feature Attribution Methods Correctly Attribute Features?](https://mlsanity.com/paper/2104.14403/) (2021)
- [Benchmarking Attribution Methods with Relative Feature Importance](https://mlsanity.com/paper/1907.09701/) (2019)
- [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019)
- [DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems](https://mlsanity.com/paper/1906.10263/) (2019)
- [Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms](https://mlsanity.com/paper/1910.07387/) (2019)
- [Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging](https://mlsanity.com/paper/2008.02766/) (2020)
- [What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors](https://mlsanity.com/paper/2009.10639/) (2020)
- [Evaluating Recurrent Neural Network Explanations](https://mlsanity.com/paper/1904.11829/) (2019)
- [Explanations for Automatic Speech Recognition](https://mlsanity.com/paper/2302.14062/) (2023)

## Built on by (most cited first, 10 of 10)

- [Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features](https://mlsanity.com/paper/2309.07733/) (2023)
- [SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation](https://mlsanity.com/paper/2411.01710/) (2024)
- [Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation](https://mlsanity.com/paper/2511.21517/) (2025)
- [Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models](https://mlsanity.com/paper/2606.14647/) (2026)
- [The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models](https://mlsanity.com/paper/2509.26543/) (2025)
- [Explainable AI in Speaker Recognition -- Attention Map Visualisation and Evaluation](https://mlsanity.com/paper/2606.22901/) (2026)
- [On the reliability of feature attribution methods for speech classification](https://mlsanity.com/paper/2505.16406/) (2025)
- [Cross-Attention is Half Explanation in Speech-to-Text Models](https://mlsanity.com/paper/2509.18010/) (2025)
- [Towards Dys-XAI: Influence-Based Explanations for Dysarthria Severity Assessment](https://mlsanity.com/paper/2606.21306/) (2026)
- [Disentangling Acoustic Cues in Alzheimer's Pathology and Perception: The Roles of Language and Gender](https://mlsanity.com/paper/2607.23977/) (2026)

## Abstract (arXiv)

Explainable AI (XAI) techniques have been widely used to help explain and understand the output of deep learning models in fields such as image classification and Natural Language Processing. Interest in using XAI techniques to explain deep learning-based automatic speech recognition (ASR) is emerging. but there is not enough evidence on whether these explanations can be trusted. To address this, we adapt a state-of-the-art XAI technique from the image classification domain, Local Interpretable Model-Agnostic Explanations (LIME), to a model trained for a TIMIT-based phoneme recognition task. This simple task provides a controlled setting for evaluation while also providing expert annotated ground truth to assess the quality of explanations. We find a variant of LIME based on time partitioned audio segments, that we propose in this paper, produces the most reliable explanations, containing the ground truth 96% of the time in its top three audio segments.