mlsanityData to 14 Oct 2026

InverseScope: Scalable Activation Inversion for Interpreting Large Language Models

arXiv 2506.07406 ↗First posted 9 Jun 2025cs.LG cs.AIcited by 2

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Understanding the internal representations of large language models (LLMs) is a central challenge in interpretability research. Existing feature interpretability methods often rely on strong structural assumptions--such as linearity or sparsity--that may not hold in practice. In this work, we introduce InverseScope, an assumption-light and scalable framework for interpreting neural activations via input inversion. Given a target activation, InverseScope characterizes its encoded information by generating natural-language inputs that produce nearby activations, grounding abstract internal states in concrete language. To overcome the prohibitive cost of sampling in high-dimensional activation spaces, we propose a novel control-layer conditioning architecture that substantially improves sample efficiency compared to prior token-prepending approaches. We demonstrate that InverseScope reveals rich geometric structure in LLM representation spaces, including sentence-level linear analogies. The framework scales to state-of-the-art open-source models of up to 14B parameters and generalizes to out-of-distribution inputs, enabling systematic analysis of activation neighborhoods.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 34 of its 46 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

34 of 46 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2014Conditional Generative Adversarial Netscited by 2,162
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  10. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  11. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  12. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
22 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  2. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  3. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  4. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  5. 2024Scaling and evaluating sparse autoencoderscited by 553
  6. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  7. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  8. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  9. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  10. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  11. 2023In-Context Learning Creates Task Vectorscited by 312
  12. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  13. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  14. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  15. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  16. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
  17. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  18. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  19. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
  20. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 51
  21. 2024Language Models Encode Numbers Using Digit Representations in Base 10cited by 32
  22. 2024InversionView: A General-Purpose Method for Reading Information from Neural Activationscited by 7

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoningcited by 8
  2. 2025Rep2Text: Decoding Full Text from a Single LLM Token Representationcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2025Superscopes: Amplifying Internal Feature Representations for Language Model Interpretation
  2. 2026A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models
  3. 2026Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior
  4. 2026WiSparse: Boosting LLM Inference Efficiency with Weight-Aware Mixed Activation Sparsity
  5. 2024Activation Scaling for Steering and Interpreting Language Models
  6. 2026One Adapter Pair per Model: A Universal Activation Interface for Language Models
  7. 2026inversedMixup: Data Augmentation via Inverting Mixed Embeddings
  8. 2026Measuring Maximum Activations in Open Large Language Models