mlsanityData to 7 Oct 2026

Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models

arXiv 2606.14647 ↗First posted 12 Jun 2026cs.SD cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Transformer-based automatic speech recognition (ASR) models such as Whisper are highly accurate, but their predictions remain difficult to interpret. Existing explainable AI (XAI) methods often lack faithfulness and precise temporal grounding. We propose Listening with Entropy-guided Attention for Faithful explainability (LEAF-X), a model-intrinsic XAI framework for transformer-based ASR. LEAF-X combines entropy-guided attention weighting, multi-layer attention rollout, and optional causal ablations to identify low-entropy, high-impact heads and layers, producing sparse token-to-frame attributions. Unlike perturbation-based explainers or raw attention maps, LEAF-X exploits the internal structure of encoder-decoder and speech-augmented decoder-only models to generate explanations that better reflect model computation. Results show 32% improved faithfulness, 35-39% stronger locality/sparsity, and the most stable attributions, supporting more transparent and auditable ASR.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

30 of 43 references.

References · 30

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  5. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  6. 2020Quantifying Attention Flow in Transformerscited by 759
  7. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  8. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  9. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  10. 2018On the Robustness of Interpretability Methodscited by 310
  11. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  12. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
18 more
  1. 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
  2. 2025Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and ASTcited by 51
  3. 2024On the Faithfulness of Vision Transformer Explanationscited by 19
  4. 2024Lost in Transcription: Identifying and Quantifying the Accuracy Biases of Automatic Speech Recognition Systems Against Disfluent Speechcited by 15
  5. 2024Measuring the Accuracy of Automatic Speech Recognition Solutionscited by 15
  6. 2023The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanationscited by 12
  7. 2023Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognitioncited by 10
  8. 2026VLA-Forget: Vision-Language-Action Unlearning for Embodied Foundation Modelscited by 9
  9. 2026Position: LLMs Must Use Functor-Based and RAG-Driven Bias Mitigation for Fairnesscited by 8
  10. 2026CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMscited by 6
  11. 2026G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMscited by 6
  12. 2024Toward Practical Automatic Speech Recognition and Post-Processing: a Call for Explainable Error Benchmark Guidelinecited by 4
  13. 2025Advances in Speech Separation: Techniques, Challenges, and Future Trendscited by 3
  14. 2026RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Modelscited by 3
  15. 2026PERSA: Reinforcement Learning for Professor-Style Personalized Feedback with LLMscited by 3
  16. 2025Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attributioncited by 2
  17. 2025Accent-Invariant Automatic Speech Recognition via Saliency-Driven Spectrogram Maskingcited by 2
  18. 2024End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026SafeCommit: Certifying When Memory-Grounded Agents May Safely Actcited by 4
  2. 2026Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMscited by 0