mlsanityData to 14 Oct 2026

Position: Behavioral Systems Require Behavioral Tests

arXiv 2608.18081 ↗First posted 31 May 2026cs.AIcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Artificial agentic systems increasingly operate as behavioral systems by interacting with dynamic environments, pursuing goals, and adapting over time. Yet, current evaluation methods largely focus on performance outcomes, not the underlying behavioral processes that produce them. This paper argues that AI agents must be evaluated like other behavioral systems: through systematic observation, perturbation, and interpretation of their actions. We draw on lessons from the behavioral sciences to motivate this position, and propose a research agenda focused on developing rigorous behavioral tests. These include methods for recovering decision strategies from action sequences, constructing environments that isolate behavioral differences, and probing emergent dynamics in multi-agent systems. Taken together, these directions offer a roadmap for developing a science of AI behavior.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 38 of its 114 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

38 of 114 references.

References · 38

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  6. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  7. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  8. 2023Augmented Language Models: a Surveycited by 403
  9. 2023Language Models can Solve Computer Taskscited by 372
  10. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  11. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 301
  12. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 201
26 more
  1. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  2. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 164
  3. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
  4. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 126
  5. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 120
  6. 2024Attacking Vision-Language Computer Agents via Pop-upscited by 116
  7. 2024Evaluating the World Model Implicit in a Generative Modelcited by 113
  8. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 113
  9. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 109
  10. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 63
  11. 2024DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Lifecited by 59
  12. 2024Large Language Models Assume People are More Rational than We Really arecited by 54
  13. 2024Measuring Implicit Bias in Explicitly Unbiased Large Language Modelscited by 51
  14. 2025Walk the Talk? Measuring the Faithfulness of Large Language Model Explanationscited by 45
  15. 2021Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policiescited by 38
  16. 2020Understanding Human Intelligence through Human Limitationscited by 33
  17. 2024Heads, Tails, and AI Fails: LLMs, Randomness, and Human Judgmentscited by 26
  18. 2024Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Functioncited by 24
  19. 2024When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1cited by 22
  20. 2023Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoMLcited by 16
  21. 2025A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experimentscited by 12
  22. 2024Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?cited by 8
  23. 2024Can we hop in general? A discussion of benchmark selection and design using the Hopper environmentcited by 7
  24. 2023An Open-Loop Baseline for Reinforcement Learning Locomotion Taskscited by 4
  25. 2025Do Large Language Models Reason Causally Like Us? Even Better?cited by 4
  26. 2025Caught in the Web of Words: Do LLMs Fall for Spin in Medical Literature?cited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Causal Behavioral Evaluation of AI Agents at Scale via Automated Behavioral Sciencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Component Testing: Validating Agentic AI Systems
  2. 2025AI Agent Behavioral Science
  3. 2026Multi-Dimensional Behavioral Evaluation of Agentic Stock Prediction Systems Using Large Language Model Judges with Closed-Loop Reinforcement Learning Feedback
  4. 2025Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems
  5. 2025Agentic AI Needs a Systems Theory
  6. 2026On Randomness in Agentic Evals
  7. 2026Towards More Standardized AI Evaluation: From Models to Agents
  8. 2026When Should an AI Act? A Human-Centered Model of Scene, Context, and Behavior for Agentic AI Design