mlsanityData to 7 Oct 2026

AI Sandbagging: Language Models can Strategically Underperform on Evaluations

arXiv 2406.07358 ↗First posted 11 Jun 2024cs.AI cs.CL cs.CYcited by 131

Not on the map. Why?

Cited by 26 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Trustworthy capability evaluations are crucial for ensuring the safety of AI systems, and are becoming a key component of AI regulation. However, the developers of an AI system, or the AI system itself, may have incentives for evaluations to understate the AI's actual capability. These conflicting interests lead to the problem of sandbagging, which we define as strategic underperformance on an evaluation. In this paper we assess sandbagging capabilities in contemporary language models (LMs). We prompt frontier LMs, like GPT-4 and Claude 3 Opus, to selectively underperform on dangerous capability evaluations, while maintaining performance on general (harmless) capability evaluations. Moreover, we find that models can be fine-tuned, on a synthetic dataset, to hide specific capabilities unless given a password. This behaviour generalizes to high-quality, held-out benchmarks such as WMDP. In addition, we show that both frontier and smaller models can be prompted or password-locked to target specific scores on a capability evaluation. We have mediocre success in password-locking a model to mimic the answers a weaker model would give. Overall, our results suggest that capability evaluations are vulnerable to sandbagging. This vulnerability decreases the trustworthiness of evaluations, and thereby undermines important safety decisions regarding the development and deployment of advanced AI systems.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Builds on 41 of its 76 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

41 of 76 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  12. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
29 more
  1. 2023Language Models Represent Space and Timecited by 306
  2. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  3. 2023Poisoning Language Models During Instruction Tuningcited by 254
  4. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  5. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  6. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  7. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 207
  8. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  9. 2023Model evaluation for extreme riskscited by 167
  10. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  11. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  12. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
  13. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 120
  14. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 110
  15. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94
  16. 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 90
  17. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 82
  18. 2023Eight Things to Know about Large Language Modelscited by 73
  19. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 63
  20. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  21. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
  22. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  23. 2023Pretraining on the Test Set Is All You Needcited by 33
  24. 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
  25. 2024Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistantcited by 32
  26. 2024The Reasons that Agents Act: Intention and Instrumental Goalscited by 23
  27. 2024Trojan Detection in Large Language Models: Insights from The Trojan Detection Challengecited by 14
  28. 2024Rethinking Generative Large Language Model Evaluation for Semantic Comprehensioncited by 13
  29. 2021Definitions of intent suitable for algorithmscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 44; 2026: 79.

131 papers.

Sort:

By area:

  1. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  2. 2024Frontier Models are Capable of In-context Schemingcited by 252
  3. 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 136
  4. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 105
  5. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agentscited by 104
  6. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  7. 2024Sabotage Evaluations for Frontier Modelscited by 45
  8. 2025A sketch of an AI control safety casecited by 36
  9. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  10. 2024Towards evaluations-based safety cases for AI schemingcited by 32
  11. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  12. 2025Evaluating Frontier Models for Stealth and Situational Awarenesscited by 30
38 more
  1. 2025Mitigating Deceptive Alignment via Self-Monitoringcited by 25
  2. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 23
  3. 2024Noise Injection Reveals Hidden Capabilities of Sandbagging Language Modelscited by 22
  4. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agentscited by 22
  5. 2025How to evaluate control measures for LLM agents? A trajectory from today to superintelligencecited by 20
  6. 2025Benchmarking Deception Probes via Black-to-White Performance Boostscited by 17
  7. 2025LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoringcited by 17
  8. 2026Intelligent AI Delegationcited by 16
  9. 2025The Elicitation Game: Evaluating Capability Elicitation Techniquescited by 15
  10. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 15
  11. 2025Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systemscited by 12
  12. 2025Difficulties with Evaluating a Deception Detector for AIscited by 11
  13. 2026SafePro: Evaluating the Safety of Professional-Level AI Agentscited by 10
  14. 2025Do Large Language Models Know What They Are Capable Of?cited by 9
  15. 2025A Survey of Theory of Mind in Large Language Models: Evaluations, Representations, and Safety Riskscited by 8
  16. 2025Safety Cases: A Scalable Approach to Frontier AI Safetycited by 8
  17. 2025Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environmentscited by 8
  18. 2025LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactionscited by 7
  19. 2025Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorscited by 7
  20. 2025AI Awarenesscited by 6
  21. 2025The Impact of Off-Policy Training Data on Probe Generalisationcited by 6
  22. 2026Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoringcited by 6
  23. 2026The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probescited by 6
  24. 2026A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoringcited by 6
  25. 20247B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancementcited by 5
  26. 2025Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methodscited by 5
  27. 2025Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Promptscited by 5
  28. 2025Relative Scaling Laws for LLMscited by 5
  29. 2026Probe-Based Data Attribution: Discovering and Mitigating Undesirable Behaviors in LLM Post-Trainingcited by 5
  30. 2026When Do LLM Preferences Predict Downstream Behavior?cited by 5
  31. 2026From Rebound to Remedy: Understanding and Mitigating Reward Hacking via Representation Engineeringcited by 5
  32. 2026Evaluation Awareness in Language Models Has Limited Effect on Behaviourcited by 5
  33. 2024Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulationcited by 4
  34. 2024What AI evaluations for preventing catastrophic risks can and cannot docited by 4
  35. 2025OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulationcited by 4
  36. 2025The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homologycited by 4
  37. 2026In-Context Environments Induce Evaluation-Awareness in Language Modelscited by 4
  38. 2026Gemma Needs Help: Investigating and Mitigating Emotional Instability in LLMscited by 4

50 most cited of 131 All 131 (JSON)