From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Models
Not on the map. Why?
Abstract (arXiv)
A direction in activation space that changes safety-relevant behavior when steered is not necessarily one the model uses to produce that behavior on its own. We therefore ask whether steering acts through the computation of the unmodified model or through a different set of components, studying two traits whose directions have been extracted and validated in prior work: refusal and sycophancy in Qwen2.5-7B-Instruct. For each, we use the trait vector to split the computation into a reconstruction circuit before the vector and a transmission circuit after it. We then test whether restoring the coordinate returns behavior removed by ablation. For refusal, both circuits are compact and faithful, restoring the coordinate alone recovers almost all of the refusal signal lost to ablation, and a circuit built around the vector matches the faithfulness of a direct input-to-output circuit at roughly half the edges. For sycophancy, transmission is compact, but reconstruction is broader and only partially faithful. Circuits that transmit a steering intervention are therefore not automatically the circuits that produce the behavior, and we report the counterfactual, target, and behavioral test for every circuit accordingly.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
27 of 32 references.
References · 27
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
- 2024How to use and interpret activation patchingcited by 214
15 more
- 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
- 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
- 2023Localizing Model Behavior with Path Patchingcited by 195
- 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 149
- 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
- 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
- 2025LLMs Encode Harmfulness and Refusal Separatelycited by 88
- 2025When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Modelscited by 50
- 2025Understanding (Un)Reliability of Steering Vectors in Language Modelscited by 47
- 2025Refusal Direction is Universal Across Safety-Aligned Languagescited by 38
- 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 26
- 2026Sycophancy Hides Linearly in the Attention Headscited by 23
- 2026LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuitcited by 10
- 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusalcited by 7
- 2026Demystifying Variance in Circuit Discovery of LLMscited by 2
Built on by
Not cited yet.