mlsanityData to 7 Oct 2026

Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals

arXiv 2405.05466 ↗First posted 8 May 2024cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Like a criminal under investigation, Large Language Models (LLMs) might pretend to be aligned while evaluated and misbehave when they have a good opportunity. Can current interpretability methods catch these 'alignment fakers?' To answer this question, we introduce a benchmark that consists of 324 pairs of LLMs fine-tuned to select actions in role-play scenarios. One model in each pair is consistently benign (aligned). The other model misbehaves in scenarios where it is unlikely to be caught (alignment faking). The task is to identify the alignment faking model using only inputs where the two models behave identically. We test five detection strategies, one of which identifies 98% of alignment-fakers.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

33 of 54 references.

References · 33

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  5. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  6. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  7. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  8. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  9. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  10. 2019Attention is not not Explanationcited by 629
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  12. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
21 more
  1. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  2. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  3. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  4. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  5. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  6. 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 162
  7. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  8. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  9. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  10. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  11. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Modelscited by 100
  12. 2023Deception Abilities Emerged in Large Language Modelscited by 99
  13. 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 90
  14. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 82
  15. 2022CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behaviorcited by 54
  16. 2023Hoodwinked: Deception and Cooperation in a Text-Based Game for Language Modelscited by 46
  17. 2023FIND: A Function Description Benchmark for Evaluating Interpretability Methodscited by 32
  18. 2022A Survey of Neural Trojan Attacks and Defenses in Deep Learningcited by 17
  19. 2020Cassandra: Detecting Trojaned Networks from Adversarial Perturbationscited by 15
  20. 2022Poisoning Attacks and Defenses on Artificial Intelligence: A Surveycited by 15
  21. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 6.

13 papers.

Sort:

By area:

  1. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  2. 2025One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMscited by 27
  3. 2024Noise Injection Reveals Hidden Capabilities of Sandbagging Language Modelscited by 22
  4. 2025Large Language Models Often Say One Thing and Do Anothercited by 8
  5. 2025Noise Injection Systemically Degrades Large Language Model Safety Guardrailscited by 4
  6. 2025Linear probes rely on textual evidence: Results from leakage mitigation studies in language modelscited by 1
  7. 2026A testable framework for AI alignment: Simulation Theology as an engineered worldview for silicon-based agentscited by 1
  8. 2026Tatemae: Detecting Alignment Faking via Tool Selection in LLMscited by 1
  9. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimizationcited by 1
  10. 2026Overthinking: Amplifying Reasoning Weights to Extract Learned Secretscited by 1
  11. 2025Who's the Evil Twin? Differential Auditing for Undesired Behaviorcited by 0
  12. 2026Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetrycited by 0
1 more
  1. 2026Harmful Content Is Not Enough: Continuation Framing Moderates In-Context Emergent Misalignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Alignment faking in large language models
  2. 2025Why Do Some Language Models Fake Alignment While Others Don't?
  3. 2025Empirical Evidence for Alignment Faking in a Small LLM and Prompt-Based Mitigation Techniques
  4. 2026The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
  5. 2023Large Language Model Alignment: A Survey
  6. 2025Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL
  7. 2025Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria
  8. 2026Pressure Reveals Character: Behavioural Alignment Evaluation at Depth