Building Better Activation Oracles
Not on the map. Why?
Abstract (arXiv)
Activation Oracles (AOs) are promising methods for interpreting residual stream activations. However, current AOs face important issues, such as hallucinations and vagueness. Additionally, text-inversion confounds make them hard to evaluate. To this end, we improve the Activation Oracle (AO) training regime in four ways: training on on-policy rollouts, improving the conversational dataset, feeding more layers and an improvement to the injection formula. The capability improvements are marginal, but quality of life improvements are quite substantial. In addition, we open source the first comprehensive evaluation suite for AO quality, which we call AObench. Overall, we hope that our work sets a foundation that helps improve AOs and other models in the paradigm of scalable, end-to-end interpretability.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 12 references that are in the map. First posted on arXiv 23 May 2026.
Builds on
6 of 12 references.
References · 6
- 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
- 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 52
- 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 51
- 2025Training Language Models to Explain Their Own Computationscited by 30
- 2025Towards eliciting latent knowledge from LLMs with mechanistic interpretabilitycited by 19
- 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistantscited by 12
Built on by
Citing papers per year: 2026: 1.
1 paper.