# AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders arXiv: https://arxiv.org/abs/2602.05027 ยท HTML: https://mlsanity.com/paper/2602.05027/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. - Builds on 38 of its 60 references that are in the map. - First posted on arXiv 4 Feb 2026. ## Placed in - Mechanistic interpretability / [SAE features and their applications](https://mlsanity.com/mechinterp/sae-features-and-their-applications/) ## Builds on (most cited first, 30 of 38) - [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022) - [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021) - [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021) - [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021) - [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019) - [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020) - [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018) - [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022) - [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015) - [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024) - [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020) - [AudioGen: Textually Guided Audio Generation](https://mlsanity.com/paper/2209.15352/) (2022) - [A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders](https://mlsanity.com/paper/2409.14507/) (2024) - [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019) - [Direct speech-to-speech translation with discrete units](https://mlsanity.com/paper/2107.05604/) (2021) - [Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders](https://mlsanity.com/paper/2410.20526/) (2024) - [SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability](https://mlsanity.com/paper/2503.09532/) (2025) - [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020) - [Textless Speech-to-Speech Translation on Real Data](https://mlsanity.com/paper/2112.08352/) (2021) - [Are Sparse Autoencoders Useful? A Case Study in Sparse Probing](https://mlsanity.com/paper/2502.16681/) (2025) - [Persona Features Control Emergent Misalignment](https://mlsanity.com/paper/2506.19823/) (2025) - [Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context](https://mlsanity.com/paper/2309.08105/) (2023) - [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025) - [RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations](https://mlsanity.com/paper/2402.17700/) (2024) - [Sparse autoencoders reveal selective remapping of visual concepts during adaptation](https://mlsanity.com/paper/2412.05276/) (2024) - [InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2412.12101/) (2024) - [A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion](https://mlsanity.com/paper/2111.02392/) (2021) - [Applying sparse autoencoders to unlearn knowledge in language models](https://mlsanity.com/paper/2410.19278/) (2024) ## Built on by (most cited first, 8 of 8) - [TADA! Tuning Audio Diffusion Models through Activation Steering](https://mlsanity.com/paper/2602.11910/) (2026) - [On the Interpretability of Whisper Encodings Using Sparse Autoencoders](https://mlsanity.com/paper/2605.12225/) (2026) - [Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography](https://mlsanity.com/paper/2605.23035/) (2026) - [Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders](https://mlsanity.com/paper/2606.10029/) (2026) - [Represented but Ignored: A Causal Account of Prosodic Underuse in Audio-Language Models](https://mlsanity.com/paper/2608.19211/) (2026) - [The Null Token Knows: Reducing Message-Free Hallucination in ASR and NMT](https://mlsanity.com/paper/2608.15940/) (2026) - [Controlling Speaking Rate in Autoregressive TTS via Activation Steering](https://mlsanity.com/paper/2609.33810/) (2026) - [From Isolated Feature to Orbits: Discovering Music Concepts via Multi-SAE Alignment](https://mlsanity.com/paper/2610.01864/) (2026) ## Abstract (arXiv) Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whisper and HuBERT, provide an extensive evaluation of their stability, interpretability, and show their practical utility. Over 50% of the features remain consistent across random seeds, and reconstruction quality is preserved. SAE features capture general acoustic and semantic information as well as specific events, including environmental noises and paralinguistic sounds (e.g. laughter, whispering) and disentangle them effectively, requiring removal of only 19-27% of features to erase a concept. Feature steering reduces Whisper's false speech detections by 70% with negligible WER increase, demonstrating real-world applicability. Finally, we find SAE features correlated with human EEG activity during speech perception, indicating alignment with human neural processing. The code and checkpoints are available at https://github.com/audiosae/audiosae_demo.