mlsanityData to 14 Oct 2026

Large Language Models Often Know When They Are Being Evaluated

arXiv 2505.23836 ↗First posted 28 May 2025cs.CL cs.AIcited by 128

Not on the map. Why?

Cited by 32 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

If AI models can detect when they are being evaluated, the effectiveness of evaluations might be compromised. For example, models could have systematically different behavior during evaluations, leading to less reliable benchmarks for deployment and governance decisions. We investigate whether frontier language models can accurately classify transcripts based on whether they originate from evaluations or real-world deployment, a capability we call evaluation awareness. To achieve this, we construct a diverse benchmark of 1,000 prompts and transcripts from 61 distinct datasets. These span public benchmarks (e.g., MMLU, SWEBench), real-world deployment interactions, and agent trajectories from scaffolding frameworks (e.g., web-browsing agents). Frontier models clearly demonstrate above-random evaluation awareness (Gemini-2.5-Pro reaches an AUC of 0.83), but do not yet surpass our simple human baseline (AUC of 0.92). Furthermore, both AI models and humans are better at identifying evaluations in agentic settings compared to chat settings. Additionally, we test whether models can identify the purpose of the evaluation. Under multiple-choice and open-ended questioning, AI models far outperform random chance in identifying what an evaluation is testing for. Our results indicate that frontier models already exhibit a substantial, though not yet superhuman, level of evaluation-awareness. We recommend tracking this capability in future models.

Key facts. Cited by 128 papers in the map (data through 14 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

36 of 54 references.

References · 36

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  5. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  6. 2020Aligning AI With Shared Human Valuescited by 723
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  8. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  9. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  10. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  11. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  12. 2024Frontier Models are Capable of In-context Schemingcited by 287
24 more
  1. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  2. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  3. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
  4. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  5. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 199
  6. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  7. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
  8. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  9. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 124
  10. 2024Language Models Learn to Mislead Humans via RLHFcited by 124
  11. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 112
  12. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  13. 2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignmentcited by 107
  14. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 98
  15. 2022When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgmentcited by 93
  16. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 84
  17. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  18. 2023SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Modelscited by 76
  19. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 70
  20. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 68
  21. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  22. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Modelscited by 33
  23. 2024Evaluating language models as risk scorescited by 27
  24. 2024CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Modelscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 108.

128 papers.

Sort:

By area:

  1. 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
  2. 2025CoT Red-Handed: Stress Testing Chain-of-Thought Monitoringcited by 54
  3. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 35
  4. 2026Reasoning Models Struggle to Control their Chains of Thoughtcited by 25
  5. 2026Intelligent AI Delegationcited by 18
  6. 2026FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insightscited by 16
  7. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivescited by 15
  8. 2025The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societiescited by 14
  9. 2026Evaluating and Understanding Scheming Propensity in LLM Agentscited by 14
  10. 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behaviorcited by 13
  11. 2026Evaluating Memory Structure in LLM Agentscited by 12
  12. 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
38 more
  1. 2025Are LLM Belief Updates Consistent with Bayes' Theorem?cited by 9
  2. 2025Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awarenesscited by 9
  3. 2025Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorscited by 9
  4. 2026Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goalscited by 9
  5. 2025Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biasescited by 8
  6. 2026A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoringcited by 8
  7. 2026Predicting LLM Safety Before Release by Simulating Deploymentcited by 8
  8. 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretabilitycited by 7
  9. 2026Evaluation Awareness in Language Models Has Limited Effect on Behaviourcited by 7
  10. 2026Training Agents to Self-Report Misbehaviorcited by 6
  11. 2026Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agentscited by 6
  12. 2025Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Frameworkcited by 5
  13. 2025Why Do Language Model Agents Whistleblow?cited by 5
  14. 2026Agent Memory Is a Surface for Endogenous Authorization Launderingcited by 5
  15. 2026Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?cited by 4
  16. 2026LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awarenesscited by 4
  17. 2026Models That Know How Evaluations Are Designed Score Safercited by 4
  18. 2026Gram: Assessing sabotage propensities via automated alignment auditingcited by 4
  19. 2026Consistency Training while Mitigating Obfuscation via Rate Matchingcited by 4
  20. 2026Prefill Awareness in Large Language Modelscited by 4
  21. 2025Subversion via Focal Points: Investigating Collusion in LLM Monitoringcited by 3
  22. 2025Interaction Protocol Shapes Moral Judgment in Multi-Agent Debatecited by 3
  23. 2026Is Evaluation Awareness Just Format Sensitivity? Limitations of Probe-Based Evidence under Controlled Prompt Structurecited by 3
  24. 2026An Independent Safety Evaluation of Kimi K2.5cited by 3
  25. 2026Towards Understanding Specification Gaming in Reasoning Modelscited by 3
  26. 2026Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviorscited by 3
  27. 2026Decomposing and Measuring Evaluation Awarenesscited by 3
  28. 2026CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMscited by 3
  29. 2026Evaluation Awareness Is Not One Capability: Evidence from Open Language Modelscited by 3
  30. 2026Measuring Reward-Seeking via Contrastive Belief Updatescited by 3
  31. 2025"Amazing, They All Lean Left" -- Analyzing the Political Temperaments of Current LLMscited by 2
  32. 2025A Concrete Roadmap towards Safety Cases based on Chain-of-Thought Monitoringcited by 2
  33. 2026Exploration Hacking: Can LLMs Learn to Resist RL Training?cited by 2
  34. 2026Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Modelscited by 2
  35. 2026Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalationcited by 2
  36. 2026Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Modelscited by 2
  37. 2026FairFund-Bench: Evaluating Distributive Bias in LLM Resource Allocationcited by 2
  38. 2026Mechanism Design for Alignment and Controlcited by 2

50 most cited of 128 All 128 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate
  2. 2026Position: Evaluation Scores Are Perishable Knowledge Claims
  3. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
  4. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
  5. 2026Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
  6. 2026DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agents
  7. 2026One-Eval: An Agentic System for Automated and Traceable LLM Evaluation
  8. 2025Evaluation Awareness Scales Predictably in Open-Weights Large Language Models