mlsanityData to 13 Oct 2026

Causal Behavioral Evaluation of AI Agents at Scale via Automated Behavioral Science

arXiv 2608.10030 ↗First posted 10 Aug 2026cs.AI cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

As AI agents are increasingly deployed in complex and new environments, knowing the conditions that influence their behavior becomes an indispensable step for their reliable and safe deployment. Yet causal behavioral evaluation of AI agents remains manual and labor-intensive. We introduce Abs2Sim and AEROBAT, a system of methods that support causal behavioral evaluation of AI agents via automated behavioral science. Given a user-specified target behavior, the methods automatically execute a full pipeline of behavioral science research---generating hypotheses about the behavior, designing and executing controlled experiments, making behavioral assessments, analyzing the results, and writing reports. For 12 target behaviors, the methods generated and tested 73 hypotheses: designing 1,160 controlled experiments and executing 22,954 simulation rounds in total. Moderate-to-strong statistical evidence emerged for 30 hypotheses, revealing potential modulators of AI behavior. In sum, our results demonstrate that automated behavioral science can extend the reach of behavioral evaluation of AI agents.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

31 of 46 references.

References · 31

  1. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 797
  2. 2023Playing repeated games with Large Language Modelscited by 216
  3. 2025Survey on Evaluation of LLM-based Agentscited by 200
  4. 2025Agentic Misalignment: How LLMs Could Be Insider Threatscited by 147
  5. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 112
  6. 2025SycEval: Evaluating LLM Sycophancycited by 107
  7. 2023Deception Abilities Emerged in Large Language Modelscited by 101
  8. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 101
  9. 2024Escalation Risks from Language Models in Military and Diplomatic Decision-Makingcited by 62
  10. 2025Position: Evaluating Generative AI Systems Is a Social Science Measurement Challengecited by 62
  11. 2024Algorithmic Collusion by Large Language Modelscited by 59
  12. 2024ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluationcited by 47
19 more
  1. 2026AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviorscited by 35
  2. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 26
  3. 2025Eliciting Language Model Behaviors with Investigator Agentscited by 23
  4. 2023The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?cited by 22
  5. 2025PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approachcited by 19
  6. 2025Lessons from a Chimp: AI "Scheming" and the Quest for Ape Languagecited by 17
  7. 2025When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmascited by 15
  8. 2026Evaluating and Understanding Scheming Propensity in LLM Agentscited by 13
  9. 2025A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experimentscited by 12
  10. 2024Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislationcited by 10
  11. 2026Propensity Inference: Environmental Contributors to LLM Behaviourcited by 7
  12. 2026Position: Anthropomorphic Misalignment Research Needs Stronger Evidencecited by 6
  13. 2025Emergence of psychopathological computations in large language modelscited by 4
  14. 2026Gram: Assessing sabotage propensities via automated alignment auditingcited by 4
  15. 2024Kiss up, Kick down: Exploring Behavioral Changes in Multi-modal Large Language Models with Assigned Visual Personascited by 3
  16. 2025Inducing State Anxiety in LLM Agents Reproduces Human-Like Biases in Consumer Decision-Makingcited by 3
  17. 2026AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluationcited by 3
  18. 2026Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalationcited by 2
  19. 2026Position: Behavioral Systems Require Behavioral Testscited by 1

Built on by

Not cited yet.