mlsanityData to 14 Oct 2026

Training LLMs to Verbalize Evaluation Awareness

arXiv 2609.36316 ↗First posted 28 Sep 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Evaluation awareness (EA) can cause large language models (LLMs) to behave differently during audits than in deployment, yet measuring and accounting for EA remains challenging. We introduce verbalization training (VT), a method for making LLMs less reticent about verbalizing evaluation awareness while avoiding to supervise the latent belief itself. VT uses a model's spontaneous verbalizations as evidence that awareness is present and truncates each rollout immediately before the verbalization, producing training prefixes at which the model is presumed to be aware. The model is then trained with an RL objective designed to increase verbalization in a calibrated way. Across Qwen3.6-35B-A3B, Kimi K2.6, and Inkling, VT increases verbalized EA by 2.4-2.9 times and transfers to held-out agentic settings, while measured latent EA and behavior remain largely stable. In a causal experiment, we independently implant meta-knowledge about evaluations through synthetic-document fine-tuning and show that VT-induced verbalizations reflect the richer knowledge acquired by the model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

25 of 44 references.

References · 25

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  3. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  4. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  5. 2024A StrongREJECT for Empty Jailbreakscited by 385
  6. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  7. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
  8. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 172
  9. 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 158
  10. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  11. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
  12. 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 86
13 more
  1. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 62
  2. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 35
  3. 2025Steering Evaluation-Aware Language Models to Act Like They Are Deployedcited by 35
  4. 2025Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoningcited by 21
  5. 2026Analyzing and Improving Chain-of-Thought Monitorability Through Information Theorycited by 10
  6. 2026Predicting LLM Safety Before Release by Simulating Deploymentcited by 8
  7. 2026Evaluation Awareness in Language Models Has Limited Effect on Behaviourcited by 7
  8. 2026Training Agents to Self-Report Misbehaviorcited by 6
  9. 2026UK AISI Alignment Evaluation Case-Studycited by 6
  10. 2026Models That Know How Evaluations Are Designed Score Safercited by 4
  11. 2026Prefill Awareness in Large Language Modelscited by 4
  12. 2026Decomposing and Measuring Evaluation Awarenesscited by 3
  13. 2026CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluation Awareness Shifts from Format to Context with Model Scale
  2. 2026Causal methods for LLM development and evaluation
  3. 2025Do Activation Verbalization Methods Convey Privileged Information?
  4. 2024Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences
  5. 2024Enhancing LLM Evaluations: The Garbling Trick
  6. 2026When Words Fall Short: Iterative Synergy Between Verbalized Reasoning and Hidden Features for LLM Confidence Estimation
  7. 2024Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate
  8. 2025How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidence