mlsanityData to 14 Oct 2026

Sparse Autoencoder Insights on Voice Embeddings

arXiv 2502.00127 ↗First posted 31 Jan 2025cs.CLcited by 1

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

Recent advances in explainable machine learning have highlighted the potential of sparse autoencoders in uncovering mono-semantic features in densely encoded embeddings. While most research has focused on Large Language Model (LLM) embeddings, the applicability of this technique to other domains remains largely unexplored. This study applies sparse autoencoders to speaker embeddings generated from a Titanet model, demonstrating the effectiveness of this technique in extracting mono-semantic features from non-textual embedded data. The results show that the extracted features exhibit characteristics similar to those found in LLM embeddings, including feature splitting and steering. The analysis reveals that the autoencoder can identify and manipulate features such as language and music, which are not evident in the original embedding. The findings suggest that sparse autoencoders can be a valuable tool for understanding and interpreting embedded data in many domains, including audio-based speaker recognition.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 11 of its 14 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

11 of 14 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Scaling and evaluating sparse autoencoders6 Jun

Other references · 10

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  5. 2017One pixel attack for fooling deep neural networkscited by 582
  6. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  7. 2021TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global contextcited by 62
  8. 2019DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systemscited by 59
  9. 2024Explainable Attribute-Based Speaker Verificationcited by 11
  10. 2020Leveraging speaker attribute information using multi task learning for speaker verification and diarizationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026On the Interpretability of Whisper Encodings Using Sparse Autoencoderscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Autoencoders Make Audio Foundation Models more Explainable
  2. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
  3. 2025Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
  4. 2025Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
  5. 2024Disentangling Dense Embeddings with Sparse Autoencoders
  6. 2025AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features
  7. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
  8. 2024Improving Dictionary Learning with Gated Sparse Autoencoders