Leverage Is Not Reach: A Control-Window Law for Single-Neuron Steering in Language Models
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
Abstract (arXiv)
Aligned language models gate behaviors such as refusal and language routing through sparse feed forward neurons, yet no theory predicts when a single neuron intervention controls a behavior coherently rather than collapsing the output. We develop a budget normalized control window framework for single neuron steering. A dose along one write direction reduces to one control coordinate: the alignment between the residual stream and the write, driven along a universal saturation curve in units of a coherence budget set by the residual norm divided by the write norm. Coherent control exists when a behavior trigger lies below the collapse ceiling. The same coordinate governs benign mode switches and refusal; the ceiling follows from weights and one generic forward pass, while triggers are measured at rollout. On fifteen held out neurons, the predicted ceiling has mean absolute error 0.14, about 0.07 in bulk layers, and the committed open or closed verdict holds on eleven against a ten of fifteen majority baseline. Closed cases expose three failure modes rather than violations: collapse before trigger, too little depth to propagate, or a normalization that caps how far one neuron can push. The law explains why local gradient attribution anti predicts control: true controllers write off the readout axis and carry a near zero first order gradient. A forward only contrastive screen made precise by the window recovers controllers that attribution misses. On refusal, the hardest case, intervention success is typed, not scalar: coherent bypass and strict actionable reach separate, so a neuron can flip refusal in fluent, on task text with no actionable content, and genuine actionable reach appears only for three of six audited Llama pivots and only at later rollout horizons. Single neuron steering is therefore a budgeted, typed audit of controllability rather than a fixed dose anecdote.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 18 of its 21 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
18 of 21 references.
Closest ancestors · 4
Same direction, newest first
- 2026A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models8 May
- 2026Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs30 Apr
- 2025NeuroStrike: Neuron-Level Attacks on Aligned LLMs15 Sep
- 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons20 Jun
Other references · 14
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2021Knowledge Neurons in Pretrained Transformerscited by 601
- 2021Causal Abstractions of Neural Networkscited by 422
2 more
- 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
- 2026Targeted Neuron Modulation via Contrastive Pair Searchcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026GAPS: Dimension-Level Gates for Conditional Activation Steering
- 2025Sparse Neuron Ablation Triggers Catastrophic Collapse of the Language Core in Large Vision-Language Models
- 2026A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models
- 2026On the Steering Dimensionality of Refusal in Language Models
- 2026Closed-Loop Neural Activation Control in Vision-Language-Action Models
- 2026How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models
- 2026Faithfulness to Refusal: A Causal Audit of Neuron Selectors
- 2026Agentic Control in Variational Language Models