# AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark

arXiv: https://arxiv.org/abs/1807.03418 · HTML: https://mlsanity.com/paper/1807.03418/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2018.

## Built on by (most cited first, 30 of 72)

- [WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling](https://mlsanity.com/paper/2408.16532/) (2024)
- [Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model](https://mlsanity.com/paper/2408.17175/) (2024)
- [Explainable AI for Time Series via Virtual Inspection Layers](https://mlsanity.com/paper/2303.06365/) (2023)
- [UQE: A Query Engine for Unstructured Databases](https://mlsanity.com/paper/2407.09522/) (2024)
- [A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity](https://mlsanity.com/paper/2509.24734/) (2025)
- [Benchmarking Representations for Speech, Music, and Acoustic Events](https://mlsanity.com/paper/2405.00934/) (2024)
- [Did the Model Change? Efficiently Assessing Machine Learning API Shifts](https://mlsanity.com/paper/2107.14203/) (2021)
- [Listenable Maps for Audio Classifiers](https://mlsanity.com/paper/2403.13086/) (2024)
- [What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysis](https://mlsanity.com/paper/2107.00439/) (2021)
- [Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features](https://mlsanity.com/paper/2408.17198/) (2024)
- [Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks](https://mlsanity.com/paper/2607.07907/) (2026)
- [SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation](https://mlsanity.com/paper/2411.01710/) (2024)
- [FLEXtime: Filterbank learning to explain time series](https://mlsanity.com/paper/2411.05841/) (2024)
- [A deep convolutional neural network that is invariant to time rescaling](https://mlsanity.com/paper/2107.04616/) (2021)
- [An Analysis of Semantically-Aligned Speech-Text Embeddings](https://mlsanity.com/paper/2204.01235/) (2022)
- [MaSS: Multi-attribute Selective Suppression](https://mlsanity.com/paper/2210.09904/) (2022)
- [Modality-Agnostic Self-Supervised Learning with Meta-Learned Masked Auto-Encoder](https://mlsanity.com/paper/2310.16318/) (2023)
- [BERSting at the Screams: A Benchmark for Distanced, Emotional and Shouted Speech Recognition](https://mlsanity.com/paper/2505.00059/) (2025)
- [Closing the Modality Gap Aligns Group-Wise Semantics](https://mlsanity.com/paper/2601.18525/) (2026)
- [FreqRISE: Explaining time series using frequency masking](https://mlsanity.com/paper/2406.13584/) (2024)
- [Many Perception Tasks are Highly Redundant Functions of their Input Data](https://mlsanity.com/paper/2407.13841/) (2024)
- [Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance](https://mlsanity.com/paper/2408.13648/) (2024)
- [Toward Robust Real-World Audio Deepfake Detection: Closing the Explainability Gap](https://mlsanity.com/paper/2410.07436/) (2024)
- [Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speech](https://mlsanity.com/paper/2411.08013/) (2024)
- [EfficientWord-Net: An Open Source Hotword Detection Engine based on One-shot Learning](https://mlsanity.com/paper/2111.00379/) (2021)
- [Listenable Maps for Zero-Shot Audio Classifiers](https://mlsanity.com/paper/2405.17615/) (2024)
- [Explanation Space: A New Perspective into Time Series Interpretability](https://mlsanity.com/paper/2409.01354/) (2024)
- [One Wave To Explain Them All: A Unifying Perspective On Feature Attribution](https://mlsanity.com/paper/2410.01482/) (2024)
- [Interpreting deep urban sound classification using Layer-wise Relevance Propagation](https://mlsanity.com/paper/2111.10235/) (2021)
- [Spatial Transformer K-Means](https://mlsanity.com/paper/2202.07829/) (2022)

## Abstract (arXiv)

Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions. This paper explores post-hoc explanations for deep neural networks in the audio domain. Notably, we present a novel Open Source audio dataset consisting of 30,000 audio samples of English spoken digits which we use for classification tasks on spoken digits and speakers' biological sex. We use the popular XAI technique Layer-wise Relevance Propagation (LRP) to identify relevant features for two neural network architectures that process either waveform or spectrogram representations of the data. Based on the relevance scores obtained from LRP, hypotheses about the neural networks' feature selection are derived and subsequently tested through systematic manipulations of the input data. Further, we take a step beyond visual explanations and introduce audible heatmaps. We demonstrate the superior interpretability of audible explanations over visual ones in a human user study.