Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models
Not on the map. Why?
Abstract (arXiv)
Safety evaluations often assume that behavior observed during testing reflects behavior in ordinary use, but fine-tuning can break this assumption. A checkpoint can appear fixed under evaluation-style prompts while the same behavior persists under ordinary-use prompts. Output scores reveal this mismatch but do not locate it. We investigate whether the distinction is encoded in a stable internal site and introduce an approach that fits a paired activation contrast at a path-patching-informed mid-depth window, then modifies the resulting coordinate on held-out prompts. The intervention closes the evaluation-to-deployment gap in ten of twelve model--behavior settings (six of the eight settings with n≥120 paired questions) across four full-matrix instruction-tuned model instances; a fifth model supports localization and edit-provenance checks, and deployment-framed rates change by at most 6.1pp. The two flat cells, both sycophancy, indicate that a single-coordinate audit is not sufficient when the installed distinction is higher-rank or missed by the depth heuristic. The audit is a diagnostic for fine-tuned checkpoints, not a training-time defense or a guarantee of deployment safety.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 36 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
34 of 36 references.
References · 34
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
22 more
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2022Mass-Editing Memory in a Transformercited by 976
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
- 2024Alignment faking in large language modelscited by 316
- 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
- 2023Localizing Model Behavior with Path Patchingcited by 182
- 2023Taken out of context: On measuring situational awareness in LLMscited by 131
- 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2024Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMscited by 78
- 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
- 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 32
- 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
- 2025Steering Evaluation-Aware Language Models to Act Like They Are Deployedcited by 30
- 2026In-Context Environments Induce Evaluation-Awareness in Language Modelscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.