# Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders

arXiv: https://arxiv.org/abs/2510.23802 · HTML: https://mlsanity.com/paper/2510.23802/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability.
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Placed in

- Mechanistic interpretability / [Activation steering methods](https://mlsanity.com/mechinterp/activation-steering-methods/)

## Builds on (most cited first, 18 of 18)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling](https://mlsanity.com/paper/2408.16532/) (2024)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model](https://mlsanity.com/paper/2408.17175/) (2024)
- [ACE-Step: A Step Towards Music Generation Foundation Model](https://mlsanity.com/paper/2506.00045/) (2025)
- [DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion](https://mlsanity.com/paper/2503.01183/) (2025)
- [Towards audio language modeling -- an overview](https://mlsanity.com/paper/2402.13236/) (2024)
- [Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2505.15957/) (2025)
- [Understanding polysemanticity in neural networks through coding theory](https://mlsanity.com/paper/2401.17975/) (2024)
- [Improving Perceptual Quality of Drum Transcription with the Expanded Groove MIDI Dataset](https://mlsanity.com/paper/2004.00188/) (2020)
- [The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling](https://mlsanity.com/paper/2209.14458/) (2022)

## Built on by (most cited first, 7 of 7)

- [Do Audio-Visual Large Language Models Really See and Hear?](https://mlsanity.com/paper/2604.02605/) (2026)
- [TADA! Tuning Audio Diffusion Models through Activation Steering](https://mlsanity.com/paper/2602.11910/) (2026)
- [Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech](https://mlsanity.com/paper/2606.01479/) (2026)
- [The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions](https://mlsanity.com/paper/2606.12629/) (2026)
- [A Quantized Native Runtime for On-Device Semantic Audio Generation](https://mlsanity.com/paper/2607.08526/) (2026)
- [Pitch-class Steering for Diffusion-based Music Generation via Latent-space Probes](https://mlsanity.com/paper/2609.04516/) (2026)
- [Tracing a Sparse Emotion-Control Circuit in LLM-Based Text-to-Speech](https://mlsanity.com/paper/2610.05080/) (2026)

## Abstract (arXiv)

While sparse autoencoders (SAEs) successfully extract interpretable features from language models, applying them to audio generation faces unique challenges: audio's dense nature requires compression that obscures semantic meaning, and automatic feature characterization remains limited. We propose a framework for interpreting audio generative models by mapping their latent representations to human-interpretable acoustic concepts. We train SAEs on audio autoencoder latents, then learn linear mappings from SAE features to discretized acoustic properties (pitch, amplitude, and timbre). This enables both controllable manipulation and analysis of the AI music generation process, revealing how acoustic properties emerge during synthesis. We validate our approach on continuous (DiffRhythm-VAE) and discrete (EnCodec, WavTokenizer) audio latent spaces, and analyze DiffRhythm, a state-of-the-art text-to-music model, to demonstrate how pitch, timbre, and loudness evolve throughout generation. While our work is only done on audio modality, our framework can be extended to interpretable analysis of visual latent space generation models.