mlsanityData to 7 Oct 2026

Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMF

arXiv 2202.11479 ↗First posted 23 Feb 2022cs.SD cs.LG eess.AScited by 23

Not on the map. Why?

Abstract (arXiv)

This paper tackles post-hoc interpretability for audio processing networks. Our goal is to interpret decisions of a network in terms of high-level audio objects that are also listenable for the end-user. To this end, we propose a novel interpreter design that incorporates non-negative matrix factorization (NMF). In particular, a carefully regularized interpreter module is trained to take hidden layer representations of the targeted network as input and produce time activations of pre-learnt NMF components as intermediate outputs. Our methodology allows us to generate intuitive audio-based interpretations that explicitly enhance parts of the input signal most relevant for a network's decision. We demonstrate our method's applicability on popular benchmarks, including a real-world multi-label classification task.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 25 of its 68 references that are in the map. First posted on arXiv 23 Feb 2022.

Builds on

25 of 68 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  6. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  7. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  8. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  9. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  10. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  11. 2016Generating Visual Explanationscited by 156
  12. 2018Learning to Separate Object Sounds by Watching Unlabeled Videocited by 114
13 more
  1. 2020Restricting the Flow: Information Bottlenecks for Attributioncited by 110
  2. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  3. 2019Explaining a black-box using Deep Variational Information Bottleneck Approachcited by 35
  4. 2019Toward Interpretable Music Tagging with Self-Attentioncited by 29
  5. 2020A Framework to Learn with Interpretationcited by 23
  6. 2020audioLIME: Listenable Explanations Using Source Separationcited by 22
  7. 2017Knowledge Transfer from Weakly Labeled Audio using Convolutional Neural Network for Sound Events and Scenescited by 20
  8. 2020Reliable Local Explanations for Machine Listeningcited by 17
  9. 2020SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Contextcited by 12
  10. 2021Tracing Back Music Emotion Predictions to Sound Sources and Intuitive Perceptual Qualitiescited by 9
  11. 2017Deep Recurrent NMF for Speech Separation by Unfolding Iterative Thresholdingcited by 3
  12. 2018Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervisioncited by 3
  13. 2020CRNNs for Urban Sound Tagging with spatiotemporal contextcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 10; 2025: 5; 2026: 4.

23 papers.

Sort:

  1. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  2. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 55
  3. 2024Listenable Maps for Audio Classifierscited by 13
  4. 2022Concept-Based Techniques for "Musicologist-friendly" Explanations in a Deep Music Classifiercited by 12
  5. 2024Focal Modulation Networks for Interpretable Sound Classificationcited by 8
  6. 2023Posthoc Interpretation via Quantizationcited by 7
  7. 2023Tackling Interpretability in Audio Classification Networks with Non-negative Matrix Factorizationcited by 5
  8. 2025Transformation of audio embeddings into interpretable, concept-based representationscited by 5
  9. 2024LMAC-TD: Producing Time Domain Explanations for Audio Classifierscited by 4
  10. 2024Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speechcited by 4
  11. 2024Listenable Maps for Zero-Shot Audio Classifierscited by 3
  12. 2024One Wave To Explain Them All: A Unifying Perspective On Feature Attributioncited by 3
11 more
  1. 2024Restyling Unsupervised Concept Based Interpretable Networks with Generative Modelscited by 2
  2. 2024Audio Explanation Synthesis with Generative Foundation Modelscited by 2
  3. 2025Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layercited by 2
  4. 2024TACO: Training-free Sound Prompted Segmentation via Semantically Constrained Audio-visual CO-factorizationcited by 1
  5. 2025From Vision to Sound: Advancing Audio Anomaly Detection with Vision-Based Algorithmscited by 1
  6. 2025From Black Box to Biomarker: Sparse Autoencoders for Interpreting Speech Models of Parkinson's Diseasecited by 1
  7. 2025Are We Merely Justifying Results ex Post Facto? Quantifying Explanatory Inversion in Post-Hoc Model Explanationscited by 0
  8. 2026TimeSAE: Causal Sparse Decoding for Faithful Explanations of Black-Box Time Series Modelscited by 0
  9. 2026AGRI-Fidelity: Evaluating the Reliability of Listenable Explanations for Poultry Disease Detectioncited by 0
  10. 2026Mask-Induced Displacement in Audio XAI via Logit Trajectory Decompositioncited by 0
  11. 2026Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanationscited by 0