# Evaluating Synthetic Activations composed of SAE Latents in GPT-2

arXiv: https://arxiv.org/abs/2409.15019 · HTML: https://mlsanity.com/paper/2409.15019/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 2 of its 17 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 2 of 2)

- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Mathematical Models of Computation in Superposition](https://mlsanity.com/paper/2408.05451/) (2024)

## Built on by (most cited first, 4 of 4)

- [Decomposing The Dark Matter of Sparse Autoencoders](https://mlsanity.com/paper/2410.14670/) (2024)
- [Characterizing stable regions in the residual stream of LLMs](https://mlsanity.com/paper/2409.17113/) (2024)
- [Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning](https://mlsanity.com/paper/2411.10397/) (2024)
- [Language Model Activations Inhabit Privileged Error-Correcting Basins](https://mlsanity.com/paper/2610.04183/) (2026)

## Abstract (arXiv)

Sparse Auto-Encoders (SAEs) are commonly employed in mechanistic interpretability to decompose the residual stream into monosemantic SAE latents. Recent work demonstrates that perturbing a model's activations at an early layer results in a step-function-like change in the model's final layer activations. Furthermore, the model's sensitivity to this perturbation differs between model-generated (real) activations and random activations. In our study, we assess model sensitivity in order to compare real activations to synthetic activations composed of SAE latents. Our findings indicate that synthetic activations closely resemble real activations when we control for the sparsity and cosine similarity of the constituent SAE latents. This suggests that real activations cannot be explained by a simple "bag of SAE latents" lacking internal structure, and instead suggests that SAE latents possess significant geometric and statistical properties. Notably, we observe that our synthetic activations exhibit less pronounced activation plateaus compared to those typically surrounding real activations.