mlsanityData to 14 Oct 2026

Auditing language models for hidden objectives

arXiv 2503.10965 ↗First posted 14 Mar 2025cs.AI cs.CL cs.LGcited by 83

Not on the map. Why?

Abstract (arXiv)

We study the feasibility of conducting alignment audits: investigations into whether models have undesired objectives. As a testbed, we train a language model with a hidden objective. Our training pipeline first teaches the model about exploitable errors in RLHF reward models (RMs), then trains the model to exploit some of these errors. We verify via out-of-distribution evaluations that the model generalizes to exhibit whatever behaviors it believes RMs rate highly, including ones not reinforced during training. We leverage this model to study alignment audits in two ways. First, we conduct a blind auditing game where four teams, unaware of the model's hidden objective or training, investigate it for concerning behaviors and their causes. Three teams successfully uncovered the model's hidden objective using techniques including interpretability with sparse autoencoders (SAEs), behavioral attacks, and training data analysis. Second, we conduct an unblinded follow-up study of eight techniques for auditing the model, analyzing their strengths and limitations. Overall, our work provides a concrete example of using alignment audits to discover a model's hidden objective and proposes a methodology for practicing and validating progress in alignment auditing.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Mar 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 58.

83 papers.

Sort:

By area:

  1. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  2. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 136
  3. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  4. 2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challengescited by 33
  5. 2025Mechanistic interpretability for steering vision-language-action modelscited by 31
  6. 2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 24
  7. 2026Model Spec Midtraining: Improving How Alignment Training Generalizescited by 20
  8. 2025Towards eliciting latent knowledge from LLMs with mechanistic interpretabilitycited by 19
  9. 2025Because we have LLMs, we Can and Should Pursue Agentic Interpretabilitycited by 16
  10. 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIscited by 14
  11. 2026Detecting Safety Violations Across Many Agent Tracescited by 14
  12. 2025Difficulties with Evaluating a Deception Detector for AIscited by 13
38 more
  1. 2026Introspection Adapters: Training LLMs to Report Their Learned Behaviorscited by 11
  2. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 10
  3. 2025Simple Mechanistic Explanations for Out-Of-Context Reasoningcited by 10
  4. 2025Reasoning Models Sometimes Output Illegible Chains of Thoughtcited by 9
  5. 2026Measuring AI R&D Automationcited by 9
  6. 2025Practical challenges of control monitoring in frontier AI deploymentscited by 8
  7. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  8. 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?cited by 8
  9. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  10. 2025Dive into the Agent Matrix: A Realistic Evaluation of Self-Replication Risk in LLM Agentscited by 7
  11. 2026Channel Location Constrains the Auditability of Subliminal Learningcited by 7
  12. 2026The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodologycited by 6
  13. 2026QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signalscited by 5
  14. 2026Positive Alignment: Artificial Intelligence for Human Flourishingcited by 5
  15. 2026Not All LLM Reasoning is Visible in the Chain-of-Thoughtcited by 5
  16. 2025Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNNcited by 4
  17. 2025Towards Effective MLLM Jailbreaking Through Balanced On-Topicness and OOD-Intensitycited by 4
  18. 2026Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluationcited by 4
  19. 2026Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?cited by 4
  20. 2026Alignment midtraining for animalscited by 4
  21. 2026Models That Know How Evaluations Are Designed Score Safercited by 4
  22. 2026Gram: Assessing sabotage propensities via automated alignment auditingcited by 4
  23. 2025Detecting Adversarial Fine-tuning with Auditing Agentscited by 3
  24. 2026Evaluating Human-AI Safety: A Framework for Measuring Harmful Capability Upliftcited by 3
  25. 2026Functional Emotions or Situational Contexts? A Discriminating Test from the Mythos Preview System Cardcited by 3
  26. 2026TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agentscited by 3
  27. 2026CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMscited by 3
  28. 2026Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoorscited by 3
  29. 2026Measuring Reward-Seeking via Contrastive Belief Updatescited by 3
  30. 2025Against racing to AGI: Cooperation, deterrence, and catastrophic riskscited by 2
  31. 2025Linear probes rely on textual evidence: Results from leakage mitigation studies in language modelscited by 2
  32. 2025ParaScopes: What do Language Models Activations Encode About Future Text?cited by 2
  33. 2026Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demandscited by 2
  34. 2026Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluationscited by 2
  35. 2025Explanations are a Means to an End: Decision Theoretic Explanation Evaluationcited by 1
  36. 2026A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agentscited by 1
  37. 2026Narrow Secret Loyalty Dodges Black-Box Auditscited by 1
  38. 2026REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinationscited by 1

50 most cited of 83 All 83 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors
  2. 2025The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives
  3. 2025Reward Auditor: Inference on Reward Modeling Suitability in Real-World Perturbed Scenarios
  4. 2024RMB: Comprehensively Benchmarking Reward Models in LLM Alignment
  5. 2026Auditing Proprietary Alignment in Large Language Models: A Comparative Framework Without a Ground-Truth Standard
  6. 2024Alignment faking in large language models
  7. 2026Alignment Forecasting: Predicting Misalignment From Training Data
  8. 2026White-Box Sensitivity Auditing with Steering Vectors