mlsanityData to 7 Oct 2026

How to Interpret Agent Behavior

arXiv 2605.13625 ↗First posted 13 May 2026cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Autonomous agents such as Claude Code and Codex now operate for hours or even days. Understanding their runtime behavior has become critical for downstream tasks such as diagnosing inefficiencies, fixing bugs, and ensuring better oversight. A primary way to gain this understanding is analyzing the reasoning trajectories and execution traces these agents generate. Yet such data remains in unstructured natural-language form, making it difficult for humans to interpret at scale. We introduce ACT*ONOMY (a combination of Action and Taxonomy), a taxonomy for describing and analyzing agent behavior at runtime. ACT*ONOMY has two components: (1) the taxonomy itself, developed through Grounded Theory and structured as a three-level hierarchy of 10 actions, 46 subactions, and 120 leaf categories; and (2) an open repository that hosts the living taxonomy, provides an automated analysis pipeline that applies it to agent trajectories analysis, and defines an extension protocol for customization and growth. Our experiments show that ACTONOMY can compare behavioral profiles across agents and characterize a single agent's behavior across diverse trajectories, surfacing patterns indicative of failure modes. By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently, enabling better oversight and control.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 45 of its 63 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

45 of 63 references.

References · 45

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  11. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  12. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
33 more
  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  2. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  4. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  5. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  6. 2024AI Agents That Mattercited by 199
  7. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  8. 2025Measuring AI Ability to Complete Long Software Taskscited by 142
  9. 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
  10. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  11. 2025Reducing Cost of LLM Agents with Trajectory Reductioncited by 44
  12. 2024CellAgent: An LLM-driven Multi-Agent Framework for Automated Single-cell Data Analysiscited by 38
  13. 2023CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Modelscited by 34
  14. 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contextscited by 26
  15. 2025REPRO-Bench: Can Agentic AI Systems Assess the Reproducibility of Social Science Research?cited by 21
  16. 2024OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Followingcited by 14
  17. 2025Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learningcited by 9
  18. 2025Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitivescited by 7
  19. 2024Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstractioncited by 6
  20. 2025ChatHLS: Towards Systematic Design Automation and Optimization for High-Level Synthesiscited by 6
  21. 2025Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocolcited by 6
  22. 2025E2Edev: Benchmarking Large Language Models in End-to-End Software Development Taskcited by 6
  23. 2026Test-Driven AI Agent Definition (TDAD): Compiling Tool-Using Agents from Behavioral Specificationscited by 6
  24. 2026Talk, Evaluate, Diagnose: User-aware Agent Evaluation with Automated Error Analysiscited by 5
  25. 2024WorldAPIs: The World Is Worth How Many APIs? A Thought Experimentcited by 4
  26. 2024My Words Imply Your Opinion: Reader Agent-based Propagation Enhancement for Personalized Implicit Emotion Analysiscited by 4
  27. 2025Text Annotation via Inductive Coding: Comparing Human Experts to LLMs in Qualitative Data Analysiscited by 4
  28. 2024Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editingcited by 3
  29. 2026GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agentscited by 3
  30. 2026ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Modelscited by 3
  31. 2026AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflowscited by 2
  32. 2026METRO: Towards Strategy Induction from Expert Dialogue Transcripts for Non-collaborative Dialoguescited by 2
  33. 2026Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interactioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Sidekick: Designing Communication for Effective Multitasking with Computer Use Agentscited by 0
  2. 2026Using Grounded Theory for Agent Behavior Analysis at Scalecited by 0
  3. 2026Rhythms of Work: Multi-Scale Interpretation of Human Behavioral Traces for Workplace Agentscited by 0
  4. 2026Your Agent Says Yes: Interpreting Adversarial Market Behavior Beyond Individual Transactionscited by 0
  5. 2026AgentPProf: Semantic Profiler for Long Horizon AI Agentscited by 0
  6. 2026On the Behavioral Traits of LLM Agentscited by 0
  7. 2026AgentHabit: Characterizing Distinct Behaviors of Agents on Everyday Taskscited by 0
  8. 2026Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflictcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Agent trajectories as programs: fingerprinting and programming coding-agent behavior
  2. 2026AgentSpy: Making AI Agent Behavior Observable
  3. 2025Agentic AI Process Observability: Discovering Behavioral Variability
  4. 2026Your Agent Has a Genome: Sequence-Level Behavioral Analysis and Runtime Governance of LLM-Powered Autonomous Agents
  5. 2026Agent ATO: Visualizing Agent Interaction Timelines from Logs
  6. 2026AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
  7. 2026From Features to Actions: Explainability in Traditional and Agentic AI Systems
  8. 2026TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces