# Beyond Transcription: Mechanistic Interpretability in ASR

arXiv: https://arxiv.org/abs/2508.15882 · HTML: https://mlsanity.com/paper/2508.15882/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability.
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Placed in

- Mechanistic interpretability / [Knowledge and representations inside LLMs](https://mlsanity.com/mechinterp/knowledge-and-representations-inside-llms/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [Linearity of Relation Decoding in Transformer Language Models](https://mlsanity.com/paper/2308.09124/) (2023)
- [Localizing Model Behavior with Path Patching](https://mlsanity.com/paper/2304.05969/) (2023)
- [AtP*: An efficient and scalable method for localizing LLM behaviour to components](https://mlsanity.com/paper/2403.00745/) (2024)
- [Transformers need glasses! Information over-squashing in language tasks](https://mlsanity.com/paper/2406.04267/) (2024)
- [Do Multilingual LLMs Think In English?](https://mlsanity.com/paper/2502.15603/) (2025)
- [Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers](https://mlsanity.com/paper/2307.03183/) (2023)
- [Detecting High-Stakes Interactions with Activation Probes](https://mlsanity.com/paper/2506.10805/) (2025)
- [Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines](https://mlsanity.com/paper/2403.05846/) (2024)
- [Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models](https://mlsanity.com/paper/2401.01572/) (2024)
- [Do Prompts Really Prompt? Exploring the Prompt Understanding Capability of Whisper](https://mlsanity.com/paper/2406.05806/) (2024)
- [A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinations](https://mlsanity.com/paper/2507.23221/) (2025)

## Built on by (most cited first, 13 of 13)

- [Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap](https://mlsanity.com/paper/2603.01502/) (2026)
- [Are Audio-Language Models Listening? Audio-Specialist Heads for Adaptive Audio Steering](https://mlsanity.com/paper/2603.06854/) (2026)
- [Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes](https://mlsanity.com/paper/2602.01247/) (2026)
- [From Text Metrics to Model Internals: A Study of Whisper ASR Hallucination Detection](https://mlsanity.com/paper/2606.23060/) (2026)
- [Scalable Music Cover Retrieval Using Lyrics-Aligned Audio Embeddings](https://mlsanity.com/paper/2601.11262/) (2026)
- [Demonstration of Adapt4Me: An Uncertainty-Aware Authoring Environment for Personalizing Automatic Speech Recognition to Non-normative Speech](https://mlsanity.com/paper/2603.20112/) (2026)
- [HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systems](https://mlsanity.com/paper/2606.23048/) (2026)
- [LOPA: Enhancing Spoken Language Assessment via Latent Ordinal Prototype Alignment](https://mlsanity.com/paper/2606.31310/) (2026)
- [CASA: Content-Acoustic Speaking Assessment with Speech Encoder and Large Language Model](https://mlsanity.com/paper/2608.13101/) (2026)
- [Towards Quantifying Benchmark Optimization in ASR Models](https://mlsanity.com/paper/2608.19936/) (2026)
- [The Anatomy of an ASR Hallucination](https://mlsanity.com/paper/2609.04404/) (2026)
- [Mechanistic Interpretability Reveals Shared Causal Subspaces in Brain-to-Speech Decoders](https://mlsanity.com/paper/2609.31992/) (2026)
- [Tracing a Sparse Emotion-Control Circuit in LLM-Based Text-to-Speech](https://mlsanity.com/paper/2610.05080/) (2026)

## Abstract (arXiv)

Interpretability methods have recently gained significant attention, particularly in the context of large language models, enabling insights into linguistic representations, error detection, and model behaviors such as hallucinations and repetitions. However, these techniques remain underexplored in automatic speech recognition (ASR), despite their potential to advance both the performance and interpretability of ASR systems. In this work, we adapt and systematically apply established interpretability methods such as logit lens, linear probing, and activation patching, to examine how acoustic and semantic information evolves across layers in ASR systems. Our experiments reveal previously unknown internal dynamics, including specific encoder-decoder interactions responsible for repetition hallucinations and semantic biases encoded deep within acoustic representations. These insights demonstrate the benefits of extending and applying interpretability techniques to speech recognition, opening promising directions for future research on improving model transparency and robustness.