mlsanityData to 7 Oct 2026

To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models

arXiv 2510.13290 ↗First posted 15 Oct 2025cs.LG cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

We introduce Mechanistic Error Reduction with Abstention (MERA), a principled framework for steering language models (LMs) to mitigate errors through selective, adaptive interventions. Unlike existing methods that rely on fixed, manually tuned steering strengths, often resulting in under or oversteering, MERA addresses these limitations by (i) optimising the intervention direction, and (ii) calibrating when, and how much to steer, thereby provably improving performance or abstaining when no confident correction is possible. Experiments across diverse datasets, and LM families demonstrate safe, effective, non-degrading error correction, and that MERA outperforms existing baselines. Moreover, MERA can be applied on top of existing steering techniques to further enhance their performance, establishing it as a general-purpose, and efficient approach to mechanistic activation steering.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

39 of 54 references.

References · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  8. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2022Toy Models of Superpositioncited by 928
  11. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  12. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
27 more
  1. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  2. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  3. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  4. 2023Function Vectors in Large Language Modelscited by 292
  5. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  6. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  7. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  8. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  9. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  10. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  11. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  12. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 74
  13. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 64
  14. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  15. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  16. 2024LoFiT: Localized Fine-tuning on LLM Representationscited by 56
  17. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  18. 2024A Language Model's Guide Through Latent Spacecited by 43
  19. 2024Multi-property Steering of Large Language Models with Dynamic Activation Compositioncited by 35
  20. 2024Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineeringcited by 33
  21. 2024Towards Reliable Evaluation of Behavior Steering Interventions in LLMscited by 33
  22. 2023Challenges with unsupervised LLM knowledge discoverycited by 30
  23. 2024LiSeCo: Linear Semantic Control for Language Generationcited by 20
  24. 2024Can sparse autoencoders be used to decompose and interpret steering vectors?cited by 19
  25. 2024Planning in a recurrent neural network that plays Sokobancited by 17
  26. 2024Towards Inference-time Category-wise Safety Steering for Large Language Modelscited by 17
  27. 2021Of Non-Linearity and Commutativity in BERTcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 26.

26 papers.

Sort:

By area:

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  2. 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Controlcited by 8
  3. 2026Steered LLM Activations are Non-Surjectivecited by 7
  4. 2026Position: Anthropomorphic Misalignment Research Needs Stronger Evidencecited by 6
  5. 2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 4
  6. 2026Steering at the Source: Style Modulation Heads for Robust Persona Controlcited by 4
  7. 2026Towards Understanding Steering Strengthcited by 3
  8. 2026Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steeringcited by 2
  9. 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Controlcited by 2
  10. 2026When is Your LLM Steerable?cited by 2
  11. 2026Deployable Per-Instance Multi-Layer Activation Steering for Large Language Modelscited by 2
  12. 2026FiLoRA: Focus-and-Ignore LoRA for Controllable Feature Reliancecited by 1
14 more
  1. 2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMscited by 1
  2. 2026Steer Like the LLM: Activation Steering that Mimics Promptingcited by 1
  3. 2026Aligning Recommendations with User Popularity Preferencescited by 0
  4. 2026Steer-to-Detect: Probing Hidden Representations for Detection of LLM-Generated Textscited by 0
  5. 2026Adaptive Probe-based Steering for Robust LLM Jailbreakingcited by 0
  6. 2026Multi-Adapter Representation Interventions via Energy Calibrationcited by 0
  7. 2026The Shape of Addition: Geometric Structures of Arithmetic in Large Language Modelscited by 0
  8. 2026Gradient-Guided Reward Optimization for Inference-time Alignmentcited by 0
  9. 2026Investigating Assistant Bias in LLM User Simulators Using a Role Vectorcited by 0
  10. 2026GAPS: Dimension-Level Gates for Conditional Activation Steeringcited by 0
  11. 2026A Probe Shift Is Not a Fairness Fix: The Limits of Representation Steering in Speech Modelscited by 0
  12. 2026Replay-Gated Neural Execution: Decoupling Persistent Behavioral Specifications from Neural Realizations in Frozen Language Modelscited by 0
  13. 2026The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstainingcited by 0
  14. 2026Reference-Tail Trust:Certified Probability Floors for Learned Updates Inside a Deployed Networkcited by 0