# Causal Behavioral Evaluation of AI Agents at Scale via Automated Behavioral Science

arXiv: https://arxiv.org/abs/2608.10030 · HTML: https://mlsanity.com/paper/2608.10030/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Builds on (most cited first, 30 of 31)

- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [Survey on Evaluation of LLM-based Agents](https://mlsanity.com/paper/2503.16416/) (2025)
- [Agentic Misalignment: How LLMs Could Be Insider Threats](https://mlsanity.com/paper/2510.05179/) (2025)
- [Measuring what Matters: Construct Validity in Large Language Model Benchmarks](https://mlsanity.com/paper/2511.04703/) (2025)
- [SycEval: Evaluating LLM Sycophancy](https://mlsanity.com/paper/2502.08177/) (2025)
- [Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://mlsanity.com/paper/2507.02825/) (2025)
- [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023)
- [Escalation Risks from Language Models in Military and Diplomatic Decision-Making](https://mlsanity.com/paper/2401.03408/) (2024)
- [Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge](https://mlsanity.com/paper/2502.00561/) (2025)
- [Algorithmic Collusion by Large Language Models](https://mlsanity.com/paper/2404.00806/) (2024)
- [ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation](https://mlsanity.com/paper/2405.14125/) (2024)
- [AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors](https://mlsanity.com/paper/2602.22755/) (2026)
- [Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents](https://mlsanity.com/paper/2502.11355/) (2025)
- [Eliciting Language Model Behaviors with Investigator Agents](https://mlsanity.com/paper/2502.01236/) (2025)
- [The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?](https://mlsanity.com/paper/2305.07970/) (2023)
- [PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach](https://mlsanity.com/paper/2511.20703/) (2025)
- [Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language](https://mlsanity.com/paper/2507.03409/) (2025)
- [When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas](https://mlsanity.com/paper/2505.19212/) (2025)
- [Evaluating and Understanding Scheming Propensity in LLM Agents](https://mlsanity.com/paper/2603.01608/) (2026)
- [A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments](https://mlsanity.com/paper/2509.25609/) (2025)
- [Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation](https://mlsanity.com/paper/2405.04325/) (2024)
- [Propensity Inference: Environmental Contributors to LLM Behaviour](https://mlsanity.com/paper/2604.21098/) (2026)
- [Position: Anthropomorphic Misalignment Research Needs Stronger Evidence](https://mlsanity.com/paper/2606.07612/) (2026)
- [Emergence of psychopathological computations in large language models](https://mlsanity.com/paper/2504.08016/) (2025)
- [Gram: Assessing sabotage propensities via automated alignment auditing](https://mlsanity.com/paper/2605.30322/) (2026)
- [Kiss up, Kick down: Exploring Behavioral Changes in Multi-modal Large Language Models with Assigned Visual Personas](https://mlsanity.com/paper/2410.03181/) (2024)
- [Inducing State Anxiety in LLM Agents Reproduces Human-Like Biases in Consumer Decision-Making](https://mlsanity.com/paper/2510.06222/) (2025)
- [AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation](https://mlsanity.com/paper/2603.07427/) (2026)
- [Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation](https://mlsanity.com/paper/2607.15434/) (2026)

## Abstract (arXiv)

As AI agents are increasingly deployed in complex and new environments, knowing the conditions that influence their behavior becomes an indispensable step for their reliable and safe deployment. Yet causal behavioral evaluation of AI agents remains manual and labor-intensive. We introduce Abs2Sim and AEROBAT, a system of methods that support causal behavioral evaluation of AI agents via automated behavioral science. Given a user-specified target behavior, the methods automatically execute a full pipeline of behavioral science research---generating hypotheses about the behavior, designing and executing controlled experiments, making behavioral assessments, analyzing the results, and writing reports. For 12 target behaviors, the methods generated and tested 73 hypotheses: designing 1,160 controlled experiments and executing 22,954 simulation rounds in total. Moderate-to-strong statistical evidence emerged for 30 hypotheses, revealing potential modulators of AI behavior. In sum, our results demonstrate that automated behavioral science can extend the reach of behavioral evaluation of AI agents.