mlsanityData to 14 Oct 2026

PCA-guided Activation Scaling for Monotonic Bidirectional Control over LLM Sycophancy

arXiv 2608.16650 ↗First posted 17 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) exhibit sycophancy, a tendency to agree with user beliefs regardless of factual accuracy. This can reinforce misconceptions, but eliminating it entirely risks over-correction against valid opinions. Effective control must therefore both reduce and increase sycophancy with predictable and gradual effect. Yet, existing methods fail to ensure a bidirectional and monotonic relationship between steering strength and behavioral outcome across models and datasets. We introduce PCA-guided Activation Scaling (PAS), an activation steering framework that decomposes residual stream activations into a PCA-identified sycophancy-honesty subspace and an orthogonal residual, then applies distinct scaling exponents to achieve monotonic, bidirectional control. Across three LLMs and three datasets, PAS achieves strong monotonicity (Spearman ρ = +0.92) and an average shift of 15.4% per direction, compared with 8.7% for the baselines. Ablation studies confirm that the decomposition, asymmetric exponents, and layer selection are each essential for maintaining monotonic control. The data and code are available at https://github.com/Bellafc/PCS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

27 of 37 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  7. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  8. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  9. 2023Steering Language Models With Activation Engineeringcited by 922
  10. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  11. 2023Whose Opinions Do Language Models Reflect?cited by 773
  12. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
15 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 243
  3. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
  4. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  5. 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
  6. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  7. 2025SycEval: Evaluating LLM Sycophancycited by 108
  8. 2024Calibrating Large Language Models with Sample Consistencycited by 73
  9. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 68
  10. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 54
  11. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
  12. 2024Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineeringcited by 39
  13. 2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 37
  14. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 35
  15. 2014Controlling Recurrent Neural Networks by Conceptorscited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Measuring Activation Control in Large Language Models
  2. 2026Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation
  3. 2026Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods
  4. 2025Steering Language Models with Weight Arithmetic
  5. 2024Linear Probe Penalties Reduce LLM Sycophancy
  6. 2025Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMs
  7. 2024Sycophancy in Large Language Models: Causes and Mitigations
  8. 2025Measuring Sycophancy of Language Models in Multi-turn Dialogues