mlsanityData to 7 Oct 2026

The Elicitation Game: Evaluating Capability Elicitation Techniques

arXiv 2502.02180 ↗First posted 4 Feb 2025cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Capability evaluations are required to understand and regulate AI systems that may be deployed or further developed. Therefore, it is important that evaluations provide an accurate estimation of an AI system's capabilities. However, in numerous cases, previously latent capabilities have been elicited from models, sometimes long after initial release. Accordingly, substantial efforts have been made to develop methods for eliciting latent capabilities from models. In this paper, we evaluate the effectiveness of capability elicitation techniques by intentionally training model organisms -- language models with hidden capabilities that are revealed by a password. We introduce a novel method for training model organisms, based on circuit-breaking, which is more robust to elicitation techniques than standard password-locked models. We focus on elicitation techniques based on prompting and activation steering, and compare these to fine-tuning methods. Prompting techniques can elicit the actual capability of both password-locked and circuit-broken model organisms in the MCQA setting, while steering fails to do so. For a code-generation task, only fine-tuning can elicit the hidden capabilities of our novel model organism. Additionally, our results suggest that combining techniques improves elicitation. Still, if possible, fine-tuning should be the method of choice to improve the trustworthiness of capability evaluations.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 36 of its 68 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

36 of 68 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
24 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 879
  2. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  3. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  4. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  5. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  6. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  7. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  8. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  9. 2023Are aligned neural networks adversarially aligned?cited by 305
  10. 2023Poisoning Language Models During Instruction Tuningcited by 254
  11. 2024Many-Shot In-Context Learningcited by 203
  12. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  13. 2023Model evaluation for extreme riskscited by 167
  14. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  15. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  16. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  17. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  18. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  19. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 110
  20. 2024Sabotage Evaluations for Frontier Modelscited by 45
  21. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
  22. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  23. 2024Trojan Detection in Large Language Models: Insights from The Trojan Detection Challengecited by 14
  24. 2024Balancing Label Quantity and Quality for Scalable Elicitationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 11.

15 papers.

Sort:

By area:

  1. 2025Distillation Robustifies Unlearningcited by 11
  2. 2026Shaping capabilities with token-level data filteringcited by 11
  3. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  4. 2026A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoringcited by 6
  5. 2025Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methodscited by 5
  6. 2025Locket: Robust Feature-Locking Technique for Language Modelscited by 1
  7. 2026LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMscited by 1
  8. 2026Modular Pretraining Enables Access Controlcited by 1
  9. 2026Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilitiescited by 1
  10. 2026A Causal Model for Locating and Unlocking Sandbagging in Model Organismscited by 1
  11. 2026Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solvercited by 0
  12. 2026AI Security Policy Should Assess Systems, Not Only Modelscited by 0
3 more
  1. 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Responsecited by 0
  2. 2026A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Revealcited by 0
  3. 2026Activation Flow: Manufacturing Activations for Steeringcited by 0