# Towards Spectroscopy: Susceptibility Clusters in Language Models

arXiv: https://arxiv.org/abs/2601.12703 · HTML: https://mlsanity.com/paper/2601.12703/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- First posted on arXiv 19 Jan 2026.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Built on by (most cited first, 4 of 4)

- [Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning](https://mlsanity.com/paper/2605.07980/) (2026)
- [Interpreting Reinforcement Learning Agents with Susceptibilities](https://mlsanity.com/paper/2605.08007/) (2026)
- [Dead Directions: Geometric Singular Learning](https://mlsanity.com/paper/2606.05957/) (2026)
- [Patterning in Practice: Debiasing Reward Models with Susceptibilities](https://mlsanity.com/paper/2609.00699/) (2026)

## Abstract (arXiv)

Spectroscopy infers the internal structure of physical systems by measuring their response to perturbations. We apply this principle to neural networks: perturbing the data distribution by upweighting a token y in context x, we measure the model's response via susceptibilities chixy, which are covariances between component-level observables and the perturbation computed over a localized Gibbs posterior via stochastic gradient Langevin dynamics (SGLD). Theoretically, we show that susceptibilities decompose as a sum over modes of the data distribution, explaining why tokens that follow their contexts "for similar reasons" cluster together in susceptibility space. Empirically, we apply this methodology to Pythia-14M, developing a conductance-based clustering algorithm that identifies 510 interpretable clusters ranging from grammatical patterns to code structure to mathematical notation. Comparing to sparse autoencoders, 50% of our clusters match SAE features, validating that both methods recover similar structure.