# Unsupervised Features Mining via Activation Geometry

arXiv: https://arxiv.org/abs/2607.04222 · HTML: https://mlsanity.com/paper/2607.04222/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 5 Jul 2026.

## Builds on (most cited first, 24 of 24)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Looking Inward: Language Models Can Learn About Themselves by Introspection](https://mlsanity.com/paper/2410.13787/) (2024)
- [Subliminal Learning: Language models transmit behavioral traits via hidden signals in data](https://mlsanity.com/paper/2507.14805/) (2025)
- [Emergent Introspective Awareness in Large Language Models](https://mlsanity.com/paper/2601.01828/) (2026)
- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers](https://mlsanity.com/paper/2512.15674/) (2025)
- [Steering Evaluation-Aware Language Models to Act Like They Are Deployed](https://mlsanity.com/paper/2510.20487/) (2025)
- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [Jailbreak Attack Initializations as Extractors of Compliance Directions](https://mlsanity.com/paper/2502.09755/) (2025)
- [Silenced Biases: The Dark Side LLMs Learned to Refuse](https://mlsanity.com/paper/2511.03369/) (2025)
- [Activation Steering for Masked Diffusion Language Models](https://mlsanity.com/paper/2512.24143/) (2025)
- [Mechanisms of Introspective Awareness](https://mlsanity.com/paper/2603.21396/) (2026)
- [Mirage Probes: How Vision Models Fake Visual Understanding](https://mlsanity.com/paper/2606.13870/) (2026)
- [Internal-State Probes Read the Situation, Not the Action: Three Negative Results for Pre-Action Misalignment Monitoring](https://mlsanity.com/paper/2606.30449/) (2026)
- [Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software](https://mlsanity.com/paper/2602.04894/) (2026)

## Abstract (arXiv)

Interpretability methods aim to reveal the features represented inside large language models (LLMs). Many existing methods begin with labeled examples of a human-defined concept that may reflect human biases, and then identify how that concept is represented within the model, for example in its activation space or through other decomposition methods. We introduce Mining via Activation Geometry (MAG), a simple unsupervised framework for extracting reasoning features from model activations by prepending the same natural-language instruction Q to every input p, where Q defines the reasoning feature of interest, such as ``Can this object be found in the desert?'' or ``Is this prompt malicious?'' We measure how the instruction changes the model's internal representation using m(Q mid p) - m(p) at a single readout point. We explore eight different MAGs. The extracted reasoning features predict the models' own world understanding and judgment, can be approximated into a single activation direction, we found that some features are more linearly represented and some less, this linear representation, which is vector steering, can change the LLMs' decisions through activation steering by injecting reasoning features. Finally, we use the same method to select the best training datasets for prompt-injection classifier probes: while similarity between ordinary activations is almost unrelated to downstream performance, RFD-based similarity achieves 94.7% Top-1 and 100% Top-2 accuracy.