Fast Multi-dimensional Refusal Subspaces via RFM-AGOP
Not on the map. Why?
Abstract (arXiv)
Steering and monitoring activations in Large Language Models (LLMs) are increasingly used for both safety and interpretability. Early work assumed behaviours are encoded along single linear directions, but recent findings suggest complex behaviours, such as the refusal to answer harmful queries, live in multi-dimensional subspaces. However, existing methods for extracting these subspaces are computationally expensive, which becomes prohibitive on reasoning models who produce long reasoning traces. By adapting the Recursive Feature Machine (RFM) algorithm -- which can be computed efficiently -- with a probe-informed initialization, we are able to identify the multi-dimensional refusal subspace in seconds, on reasoning (Qwen 3) and non-reasoning (Qwen 2.5) models. While RFM allows for faster subspace identification, it also showed better performances on the ablation task than its alternatives. More work is planned to better understand the relations between subspaces found by different methods. If confirmed, RFM could be a cheap and scalable complement to existing subspace-extraction methods in LLMs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 2 Jul 2026.
Builds on
16 of 25 references.
References · 16
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2024A StrongREJECT for Empty Jailbreakscited by 385
- 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
- 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 159
- 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 60
- 2022Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn featurescited by 25
- 2025COSMIC: Generalized Refusal Direction Identification in LLM Activationscited by 19
- 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
- 2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectivescited by 8
4 more
- 2025Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMscited by 8
- 2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 8
- 2025Steering Autoregressive Music Generation with Recursive Feature Machinescited by 6
- 2026How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Non-linear Interventions on Large Language Models
- 2025Verifying Large Language Models' Reasoning Paths via Correlation Matrix Rank
- 2026On the Steering Dimensionality of Refusal in Language Models
- 2024CARL-GT: Evaluating Causal Reasoning Capabilities of Large Language Models
- 2026RISA: Response Inspection and Selective Actions for Refusal Calibration in Large Language Models
- 2026Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
- 2025Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topics
- 2026Discovering a Shared Logical Subspace: Steering LLM Logical Reasoning via Alignment of Natural-Language and Symbolic Views