# AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

arXiv: https://arxiv.org/abs/2602.22755 · HTML: https://mlsanity.com/paper/2602.22755/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2026.

## Built on by (most cited first, 30 of 38)

- [Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges](https://mlsanity.com/paper/2604.13602/) (2026)
- [Detecting Safety Violations Across Many Agent Traces](https://mlsanity.com/paper/2604.11806/) (2026)
- [Introspection Adapters: Training LLMs to Report Their Learned Behaviors](https://mlsanity.com/paper/2604.16812/) (2026)
- [Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale](https://mlsanity.com/paper/2605.20744/) (2026)
- [Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence](https://mlsanity.com/paper/2604.08169/) (2026)
- [Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?](https://mlsanity.com/paper/2604.11061/) (2026)
- [Models That Know How Evaluations Are Designed Score Safer](https://mlsanity.com/paper/2605.28591/) (2026)
- ["Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms](https://mlsanity.com/paper/2606.12618/) (2026)
- [Why Do AI Agents Break Rules? How Framing, Context, and Social Signals Shape Compliance](https://mlsanity.com/paper/2608.12323/) (2026)
- [Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands](https://mlsanity.com/paper/2605.15164/) (2026)
- [Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing](https://mlsanity.com/paper/2606.17478/) (2026)
- [ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D](https://mlsanity.com/paper/2607.19321/) (2026)
- [Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments](https://mlsanity.com/paper/2608.16747/) (2026)
- [Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals](https://mlsanity.com/paper/2605.26045/) (2026)
- [PreAct-Bench: Benchmarking Predictive Monitoring in LLMs](https://mlsanity.com/paper/2606.09890/) (2026)
- [Revealing Hidden Model Behaviors with Task-Specific Self-Reports](https://mlsanity.com/paper/2607.03640/) (2026)
- [ToxScreen: Detecting Whether an LLM Has Been Poisoned](https://mlsanity.com/paper/2607.26849/) (2026)
- [From Deceptive Outputs to Deceptive Mechanisms: A Causal Framework for Language-Model Deception Research](https://mlsanity.com/paper/2609.04166/) (2026)
- [AISafetyBenchExplorer: A Metric-Aware Catalogue of AI Safety Benchmarks Reveals Fragmented Measurement and Weak Benchmark Governance](https://mlsanity.com/paper/2604.12875/) (2026)
- [Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations](https://mlsanity.com/paper/2604.22109/) (2026)
- [Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives](https://mlsanity.com/paper/2605.00994/) (2026)
- [The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent Misalignment](https://mlsanity.com/paper/2606.10747/) (2026)
- [Fuzzing Large Language Models to Elicit Hidden Behaviours](https://mlsanity.com/paper/2606.29646/) (2026)
- [Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation](https://mlsanity.com/paper/2607.01208/) (2026)
- [Causal Behavioral Evaluation of AI Agents at Scale via Automated Behavioral Science](https://mlsanity.com/paper/2608.10030/) (2026)
- [Evaluating RL Explainability Methods by How Much They Help Fix Bugs in Agents](https://mlsanity.com/paper/2608.17524/) (2026)
- [Training Alignment Auditors via Reinforcement Learning](https://mlsanity.com/paper/2608.25460/) (2026)
- [BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing](https://mlsanity.com/paper/2608.31105/) (2026)
- [SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?](https://mlsanity.com/paper/2609.09113/) (2026)
- [Shallow Beliefs: Synthetic document finetuning does not inoculate against emergent misalignment from reward hacking](https://mlsanity.com/paper/2609.14998/) (2026)

## Abstract (arXiv)

We introduce AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors. Each model has one of 14 concerning behaviors--such as sycophantic deference, opposition to AI regulation, or secret geopolitical loyalties--which it does not confess to when directly asked. AuditBench models are highly diverse--some are subtle, while others are overt, and we use varying training techniques both for implanting behaviors and training models not to confess. To demonstrate AuditBench's utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools. By measuring investigator agent success using different tools, we can evaluate their efficacy. Notably, we observe a tool-to-agent gap, where tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent. We find that our most effective tools involve scaffolded calls to auxiliary models that generate diverse prompts for the target. White-box interpretability tools can be helpful, but the agent performs best with black-box tools. We also find that audit success varies greatly across training techniques: models trained on synthetic documents are easier to audit than models trained on demonstrations, with better adversarial training further increasing auditing difficulty. We release our models, agent, and evaluation framework to support future quantitative, iterative science on alignment auditing.