mlsanityData to 13 Oct 2026

Learning Correct Behavior from Examples: Validating Sequential Execution in Autonomous Agents

arXiv 2605.03159 ↗First posted 4 May 2026cs.AI cs.SEcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

As autonomous agents become increasingly sophisticated, validating their sequential behavior presents a significant challenge. Traditional testing approaches require manual specification, exact sequence matching, or thousands of training examples. We present a novel algorithm that automatically learns correct behavior from just 2-10 passing execution traces and validates new executions against this learned model. Our approach combines dominator analysis from compiler theory with multimodal large language model-powered semantic understanding to identify essential states and handle non-deterministic behavior. The system constructs a generalized ground truth model using Prefix Tree Acceptors, merges traces through multi-tiered equivalence detection, and validates new executions via topological subsequence matching. In controlled experiments, our system achieved high accuracy in detecting product bugs and false successes using only 3 training traces. This approach provides explainable validation results with coverage metrics and works across diverse domains including UI testing, code generation, and robotic processes.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 32 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

9 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Willful Disobedience: Automatically Detecting Failures in Agentic Traces25 Mar
  2. 2026ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Files28 Feb
  3. 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectories2 Feb

Other references · 6

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,274
  3. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,992
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,218
  5. 2024The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Usecited by 62
  6. 2021Using Machine Learning to Generate Test Oracles: A Systematic Literature Reviewcited by 7

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025An Approach to Checking Correctness for Agentic Systems
  2. 2026AutoRefine: Compiling Trajectories into Validated Typed Agent Artifacts
  3. 2026TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces
  4. 2026TriCEGAR: A Trace-Driven Abstraction Mechanism for Agentic AI
  5. 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents
  6. 2026Your Agent Has a Genome: Sequence-Level Behavioral Analysis and Runtime Governance of LLM-Powered Autonomous Agents
  7. 2026PBT-Bench: Benchmarking AI Agents on Property-Based Testing
  8. 2026Agentic Fuzzing: Opportunities and Challenges