mlsanityData to 7 Oct 2026

AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders

arXiv 2602.05027 ↗First posted 4 Feb 2026cs.SD cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whisper and HuBERT, provide an extensive evaluation of their stability, interpretability, and show their practical utility. Over 50% of the features remain consistent across random seeds, and reconstruction quality is preserved. SAE features capture general acoustic and semantic information as well as specific events, including environmental noises and paralinguistic sounds (e.g. laughter, whispering) and disentangle them effectively, requiring removal of only 19-27% of features to erase a concept. Feature steering reduces Whisper's false speech detections by 70% with negligible WER increase, demonstrating real-world applicability. Finally, we find SAE features correlated with human EEG activity during speech perception, indicating alignment with human neural processing. The code and checkpoints are available at https://github.com/audiosae/audiosae_demo.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

38 of 60 references.

References · 38

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  7. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  9. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  10. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  11. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 400
  12. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
26 more
  1. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  2. 2022AudioGen: Textually Guided Audio Generationcited by 292
  3. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 154
  4. 2021Direct speech-to-speech translation with discrete unitscited by 148
  5. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  6. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
  7. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  8. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
  9. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  10. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  11. 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 84
  12. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  13. 2025Persona Features Control Emergent Misalignmentcited by 76
  14. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  15. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptationcited by 66
  16. 2024InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoderscited by 64
  17. 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
  18. 2024Applying sparse autoencoders to unlearn knowledge in language modelscited by 58
  19. 2024Mathematical Models of Computation in Superpositioncited by 34
  20. 2025Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audiocited by 31
  21. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion modelscited by 29
  22. 2024Disentangling Dense Embeddings with Sparse Autoencoderscited by 19
  23. 2024Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Modelscited by 15
  24. 2025Feature-Level Insights into Artificial Text Detection with Sparse Autoencoderscited by 14
  25. 2025Discovering and Steering Interpretable Concepts in Large Generative Music Modelscited by 11
  26. 2023TaCo: Targeted Concept Erasure Prevents Non-Linear Classifiers From Detecting Protected Attributescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026TADA! Tuning Audio Diffusion Models through Activation Steeringcited by 2
  2. 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topographycited by 2
  3. 2026On the Interpretability of Whisper Encodings Using Sparse Autoencoderscited by 1
  4. 2026Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoderscited by 1
  5. 2026Represented but Ignored: A Causal Account of Prosodic Underuse in Audio-Language Modelscited by 1
  6. 2026The Null Token Knows: Reducing Message-Free Hallucination in ASR and NMTcited by 0
  7. 2026Controlling Speaking Rate in Autoregressive TTS via Activation Steeringcited by 0
  8. 2026From Isolated Feature to Orbits: Discovering Music Concepts via Multi-SAE Alignmentcited by 0