# Listenable Maps for Audio Classifiers

arXiv: https://arxiv.org/abs/2403.13086 · HTML: https://mlsanity.com/paper/2403.13086/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Evaluating and Aggregating Feature-based Model Explanations](https://mlsanity.com/paper/2005.00631/) (2020)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [Explaining Image Classifiers by Counterfactual Generation](https://mlsanity.com/paper/1807.08024/) (2018)
- [Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks](https://mlsanity.com/paper/1801.03454/) (2018)
- [AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark](https://mlsanity.com/paper/1807.03418/) (2018)
- [Toward Interpretable Music Tagging with Self-Attention](https://mlsanity.com/paper/1906.04972/) (2019)
- [Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMF](https://mlsanity.com/paper/2202.11479/) (2022)
- [audioLIME: Listenable Explanations Using Source Separation](https://mlsanity.com/paper/2008.00582/) (2020)
- [Reliable Local Explanations for Machine Listening](https://mlsanity.com/paper/2005.07788/) (2020)
- [Tracing Back Music Emotion Predictions to Sound Sources and Intuitive Perceptual Qualities](https://mlsanity.com/paper/2106.07787/) (2021)
- [Classifier-agnostic saliency map extraction](https://mlsanity.com/paper/1805.08249/) (2018)
- [Investigating and Simplifying Masking-based Saliency Methods for Model Interpretability](https://mlsanity.com/paper/2010.09750/) (2020)

## Built on by (most cited first, 13 of 13)

- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [Innovative Speech-Based Deep Learning Approaches for Parkinson's Disease Classification: A Systematic Review](https://mlsanity.com/paper/2407.17844/) (2024)
- [LMAC-TD: Producing Time Domain Explanations for Audio Classifiers](https://mlsanity.com/paper/2409.08655/) (2024)
- [Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speech](https://mlsanity.com/paper/2411.08013/) (2024)
- [Listenable Maps for Zero-Shot Audio Classifiers](https://mlsanity.com/paper/2405.17615/) (2024)
- [One Wave To Explain Them All: A Unifying Perspective On Feature Attribution](https://mlsanity.com/paper/2410.01482/) (2024)
- [A Data-Driven Diffusion-based Approach for Audio Deepfake Explanations](https://mlsanity.com/paper/2506.03425/) (2025)
- [From Vision to Sound: Advancing Audio Anomaly Detection with Vision-Based Algorithms](https://mlsanity.com/paper/2502.18328/) (2025)
- [From Black Box to Biomarker: Sparse Autoencoders for Interpreting Speech Models of Parkinson's Disease](https://mlsanity.com/paper/2507.16836/) (2025)
- [Sparse Autoencoders Make Audio Foundation Models more Explainable](https://mlsanity.com/paper/2509.24793/) (2025)
- [Aligned explanations in neural networks](https://mlsanity.com/paper/2601.04378/) (2026)
- [Mask-Induced Displacement in Audio XAI via Logit Trajectory Decomposition](https://mlsanity.com/paper/2609.33486/) (2026)
- [Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanations](https://mlsanity.com/paper/2609.34030/) (2026)

## Abstract (arXiv)

Despite the impressive performance of deep learning models across diverse tasks, their complexity poses challenges for interpretation. This challenge is particularly evident for audio signals, where conveying interpretations becomes inherently difficult. To address this issue, we introduce Listenable Maps for Audio Classifiers (L-MAC), a posthoc interpretation method that generates faithful and listenable interpretations. L-MAC utilizes a decoder on top of a pretrained classifier to generate binary masks that highlight relevant portions of the input audio. We train the decoder with a loss function that maximizes the confidence of the classifier decision on the masked-in portion of the audio while minimizing the probability of model output for the masked-out portion. Quantitative evaluations on both in-domain and out-of-domain data demonstrate that L-MAC consistently produces more faithful interpretations than several gradient and masking-based methodologies. Furthermore, a user study confirms that, on average, users prefer the interpretations generated by the proposed technique.