# Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts

arXiv: https://arxiv.org/abs/2511.06048 · HTML: https://mlsanity.com/paper/2511.06048/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability.
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 8 Nov 2025.

## Placed in

- Mechanistic interpretability / [SAE features and their applications](https://mlsanity.com/mechinterp/sae-features-and-their-applications/)

## Builds on (most cited first, 30 of 33)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024)
- [Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders](https://mlsanity.com/paper/2407.14435/) (2024)
- [AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders](https://mlsanity.com/paper/2501.17148/) (2025)
- [k-Sparse Autoencoders](https://mlsanity.com/paper/1312.5663/) (2013)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [Knowledge Editing for Large Language Models: A Survey](https://mlsanity.com/paper/2310.16218/) (2023)
- [Learning Multi-Level Features with Matryoshka Sparse Autoencoders](https://mlsanity.com/paper/2503.17547/) (2025)
- [SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability](https://mlsanity.com/paper/2503.09532/) (2025)
- [Automatically Interpreting Millions of Features in Large Language Models](https://mlsanity.com/paper/2410.13928/) (2024)
- [Sparse Autoencoders Trained on the Same Data Learn Different Features](https://mlsanity.com/paper/2501.16615/) (2025)
- [Improving Steering Vectors by Targeting Sparse Autoencoder Features](https://mlsanity.com/paper/2411.02193/) (2024)
- [Sparse Autoencoders Do Not Find Canonical Units of Analysis](https://mlsanity.com/paper/2502.04878/) (2025)
- [A survey of sparse representation: algorithms and applications](https://mlsanity.com/paper/1602.07017/) (2016)
- [WizMap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings](https://mlsanity.com/paper/2306.09328/) (2023)
- [Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders](https://mlsanity.com/paper/2411.01220/) (2024)
- [Mechanistic Permutability: Match Features Across Layers](https://mlsanity.com/paper/2410.07656/) (2024)
- [SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs](https://mlsanity.com/paper/2504.08192/) (2025)
- [Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video](https://mlsanity.com/paper/2504.19475/) (2025)
- [Residual Stream Analysis with Multi-Layer SAEs](https://mlsanity.com/paper/2409.04185/) (2024)
- [Evolution of SAE Features Across Layers in LLMs](https://mlsanity.com/paper/2410.08869/) (2024)
- [Mapper Interactive: A Scalable, Extendable, and Interactive Toolbox for the Visual Exploration of High-Dimensional Data](https://mlsanity.com/paper/2011.03209/) (2020)
- [Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships](https://mlsanity.com/paper/2407.12543/) (2024)
- [Embedding Atlas: Low-Friction, Interactive Embedding Visualization](https://mlsanity.com/paper/2505.06386/) (2025)

## Built on by (most cited first, 2 of 2)

- [Interpretable and Steerable Concept Bottleneck Sparse Autoencoders](https://mlsanity.com/paper/2512.10805/) (2025)
- [Sparse Autoencoders as a Steering Basis for Phase Synchronization in Graph-Based CFD Surrogates](https://mlsanity.com/paper/2604.04946/) (2026)

## Abstract (arXiv)

Sparse autoencoders (SAEs) have emerged as a powerful tool for uncovering interpretable features in large language models (LLMs) through the sparse directions they learn. However, the sheer number of extracted directions makes comprehensive exploration intractable. While conventional embedding techniques such as UMAP can reveal global structure, they suffer from limitations including high-dimensional compression artifacts, overplotting, and misleading neighborhood distortions. In this work, we propose a focused exploration framework that prioritizes curated concepts and their corresponding SAE features over attempts to visualize all available features simultaneously. We present an interactive visualization system that combines topology-based visual encoding with dimensionality reduction to faithfully represent both local and global relationships among selected features. This hybrid approach enables users to investigate SAE behavior through targeted, interpretable subsets, facilitating deeper and more nuanced analysis of concept representation in latent space.