mlsanityData to 7 Oct 2026

Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment

arXiv 2608.11528 ↗First posted 12 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Group alignment adapts a language model to a demographic group to produce responses that reflect the group's opinions, values, and preferences. Sycophancy, a well-documented by-product of alignment, causes the model to over-agree with the user regardless of factual and objective information. However, existing group alignment methods and evaluations focus only on how closely the model matches the group's opinions, overlooking the induced change in sycophantic behaviour. To bridge this gap, we introduce Group Alignment-induced Sycophancy (GAS) and systematically evaluate alignment across 3 methods, 4 models and 13 demographic groups, on both the intended gain in opinion alignment and the unintended shift in sycophancy. We find that gain and shift are non-uniform across groups: under an identical budget, some groups receive larger gains in opinion alignment than others, and the induced sycophancy shift forms a group-specific profile rather than a single-dimensional change. These results suggest that group alignment should be reported as a two-sided, multi-dimensional profile rather than a single fit score that accounts for per-group differences when adapting LLMs to diverse populations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

28 of 34 references.

References · 28

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  3. 2023Whose Opinions Do Language Models Reflect?cited by 759
  4. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  5. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  6. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  7. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  8. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
  9. 2024A Roadmap to Pluralistic Alignmentcited by 175
  10. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
  11. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  12. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
16 more
  1. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaborationcited by 111
  2. 2023Aligning Language Models to User Opinionscited by 98
  3. 2024Group Robust Preference Optimization in Reward-free RLHFcited by 94
  4. 2025Measuring Sycophancy of Language Models in Multi-turn Dialoguescited by 91
  5. 2023Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Promptingcited by 80
  6. 2024Evaluating Large Language Model Biases in Persona-Steered Generationcited by 78
  7. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 51
  8. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 40
  9. 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Datasetcited by 32
  10. 2025BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMscited by 23
  11. 2024No Preference Left Behind: Group Distributional Preference Optimizationcited by 21
  12. 2025Interaction Context Often Increases Sycophancy in LLMscited by 21
  13. 2026OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agentscited by 18
  14. 2026MemSyco-Bench: Benchmarking Sycophancy in Agent Memorycited by 9
  15. 2026Verbalizing LLMs' assumptions to explain and control sycophancycited by 3
  16. 2025Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignmentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior
  2. 2026When Helpfulness Becomes Sycophancy: Sycophancy is a Boundary Failure Between Social Alignment and Epistemic Integrity in Large Language Models
  3. 2026CommunityBench: Benchmarking Community-Level Alignment across Diverse Groups and Tasks
  4. 2026Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models
  5. 2024Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensiveness
  6. 2026Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions
  7. 2026Sycophantic Agreement Transfers with Neutral Data via Contrastive Preference Optimization
  8. 2024Benchmarking Distributional Alignment of Large Language Models