# Tackling Polysemanticity with Neuron Embeddings

arXiv: https://arxiv.org/abs/2411.08166 · HTML: https://mlsanity.com/paper/2411.08166/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 5 of 5)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [An Interpretability Illusion for BERT](https://mlsanity.com/paper/2104.07143/) (2021)
- [Neuron to Graph: Interpreting Language Model Neurons at Scale](https://mlsanity.com/paper/2305.19911/) (2023)

## Built on by (most cited first, 2 of 2)

- [Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence](https://mlsanity.com/paper/2505.11611/) (2025)
- [How Many Thoughts Can a Vector Hold? The Capacity of Reasoning by Superposition](https://mlsanity.com/paper/2609.13747/) (2026)

## Abstract (arXiv)

We present neuron embeddings, a representation that can be used to tackle polysemanticity by identifying the distinct semantic behaviours in a neuron's characteristic dataset examples, making downstream manual or automatic interpretation much easier. We apply our method to GPT2-small, and provide a UI for exploring the results. Neuron embeddings are computed using a model's internal representations and weights, making them domain and architecture agnostic and removing the risk of introducing external structure which may not reflect a model's actual computation. We describe how neuron embeddings can be used to measure neuron polysemanticity, which could be applied to better evaluate the efficacy of Sparse Auto-Encoders (SAEs).