# Language Model Activations Inhabit Privileged Error-Correcting Basins

arXiv: https://arxiv.org/abs/2610.04183 · HTML: https://mlsanity.com/paper/2610.04183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 3 Oct 2026.

## Builds on (most cited first, 29 of 29)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [Extracting Latent Steering Vectors from Pretrained Language Models](https://mlsanity.com/paper/2205.05124/) (2022)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence](https://mlsanity.com/paper/2502.17420/) (2025)
- [Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories](https://mlsanity.com/paper/2406.00034/) (2024)
- [Controlling Language and Diffusion Models by Transporting Activations](https://mlsanity.com/paper/2410.23054/) (2024)
- [Attractor and integrator networks in the brain](https://mlsanity.com/paper/2112.03978/) (2021)
- [Explorations of Self-Repair in Language Models](https://mlsanity.com/paper/2402.15390/) (2024)
- [Mathematical Models of Computation in Superposition](https://mlsanity.com/paper/2408.05451/) (2024)
- [To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models](https://mlsanity.com/paper/2510.13290/) (2025)
- [Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior](https://mlsanity.com/paper/2605.05115/) (2026)
- [Curveball Steering: The Right Direction To Steer Isn't Always Linear](https://mlsanity.com/paper/2603.09313/) (2026)
- [Dynamically Scaled Activation Steering](https://mlsanity.com/paper/2512.03661/) (2025)
- [ReCoVeR the Target Language: Language Steering without Sacrificing Task Performance](https://mlsanity.com/paper/2509.14814/) (2025)
- [FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers](https://mlsanity.com/paper/2605.17231/) (2026)
- [Characterizing stable regions in the residual stream of LLMs](https://mlsanity.com/paper/2409.17113/) (2024)
- [Evaluating Synthetic Activations composed of SAE Latents in GPT-2](https://mlsanity.com/paper/2409.15019/) (2024)
- [Mechanistically Eliciting Latent Behaviors in Language Models](https://mlsanity.com/paper/2606.29604/) (2026)
- [Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering](https://mlsanity.com/paper/2605.24942/) (2026)
- [Evidence for feature-specific error correction in LLMs](https://mlsanity.com/paper/2606.24964/) (2026)

## Abstract (arXiv)

Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize that this robustness is a result of passive dynamics, i.e., constraining mechanisms in the forward pass that funnel activations toward "good" regions that produce coherent outputs. To investigate these hypothesized error-correcting mechanisms, we probe the geometry of language model activation space by observing the action of model layers on low-dimensional curves. In doing so, we discover that model activations occur within a cluster of distinct attracting basins, which differentiate natural activations geometrically from distributionally similar synthetic activations. Applying this lens to language model steering, we observe feature-specific basins along semantic steering directions, and find that steering moves activations between these basins. To demonstrate the active role of this geometry in neural computation, we show that adaptively modulating steering strength to transport activations across basins improves inter-language steering, significantly increasing the probability of sampling tokens from the target language compared to fixed-strength steering. Our findings establish analysis of activation space geometry as a promising approach to interpreting and controlling language models.