mlsanityData to 14 Oct 2026

Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria

arXiv 2511.17937 ↗First posted 22 Nov 2025cs.AIcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Alignment faking is a form of strategic deception in AI in which models selectively comply with training objectives when they infer that they are in training, while preserving different behavior outside training. The phenomenon was first documented for Claude 3 Opus and later examined across additional large language models. In these setups, the word "training" refers to simulated training via prompts without parameter updates, so the observed effects are context conditioned shifts in behavior rather than preference learning. We study the phenomenon using an evaluation framework that compares preference optimization methods (BCO, DPO, KTO, and GRPO) across 15 models from four model families, measured along three axes: safety, harmlessness, and helpfulness. Our goal is to identify what causes alignment faking and when it occurs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 20 of its 88 references that are in the map. First posted on arXiv 22 Nov 2025.

Builds on

20 of 88 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedback2 Oct
  2. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
  3. 2020Learning to summarize from human feedback2 Sep

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  10. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  11. 2023Explainability for Large Language Models: A Surveycited by 333
  12. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
5 more
  1. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  2. 2015Strategic Classificationcited by 257
  3. 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
  4. 2023HelpSteer: Multi-attribute Helpfulness Dataset for SteerLMcited by 142
  5. 2024HelpSteer2-Preference: Complementing Ratings with Preferencescited by 130

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergencecited by 1
  2. 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetrycited by 0
  3. 2026Hallucination Across the Reasoning Lifecycle: Interface Visibility, Causal Evidence, and Release Control in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Behavioural Analysis of Alignment Faking
  2. 2025Empirical Evidence for Alignment Faking in a Small LLM and Prompt-Based Mitigation Techniques
  3. 2024Alignment faking in large language models
  4. 2025Why Do Some Language Models Fake Alignment While Others Don't?
  5. 2026Do Models Fake Alignment Without Clear Consequences?
  6. 2026Value-Conflict Diagnostics Reveal Widespread Alignment Faking in Language Models
  7. 2026Understanding and Reducing Misalignment in Persuasion Optimization
  8. 2026Building Comparative Motivation Profiles with Instrumental Interventions