mlsanityData to 14 Oct 2026

PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors

arXiv 2605.06455 ↗First posted 7 May 2026cs.AIcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large language model (LLM) agents execute multi-step tasks, but terminal verdicts arrive too late for intervention. Agent traces mix messages, tool calls, and feedback, making it difficult to learn failure signals from terminal outcomes. Even if a model learns to predict failure from execution prefixes, its predictions alone cannot show which actions, feedback, or unmet conditions need inspection. Existing work studies online failure prediction and diagnosis from completed trajectories, yet few methods connect timely warning to a compact symbolic model of execution behavior. We introduce PrefixGuard, a neuro-symbolic method that uses learned events to connect online failure warning with finite-state execution diagnosis. A gated recurrent unit (GRU) predicts horizon-specific failure risk from these events. A deterministic finite automaton (DFA) organizes their histories into compact risk-labeled paths for replay and rule-monitor composition. Across all 24 benchmark--horizon settings, PrefixGuard exceeds the strongest evaluated baseline in mean test area under the precision--recall curve (AUPRC). Average gains range from 12.0 to 21.9 percentage points. At a nominal 10% FAR budget, averaged test recall spans 56.1%--93.0% across benchmarks. On one frozen τ2-Bench DFA, model checking five specifications identifies violations in 21 of 265 benchmark-successful runs.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 34 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

18 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 16

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  5. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  6. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  7. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
  8. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 137
  9. 2021LogBERT: Log Anomaly Detection via BERTcited by 118
  10. 2016Predictive Business Process Monitoring with LSTM Neural Networkscited by 104
  11. 2017Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamplescited by 66
  12. 2020Interpretable Sequence Classification via Discrete Optimizationcited by 19
4 more
  1. 2019Fire Now, Fire Later: Alarm-Based Systems for Prescriptive Process Monitoringcited by 18
  2. 2024DeepDFA: Automata Learning through Neural Probabilistic Relaxationscited by 8
  3. 2024Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extractioncited by 8
  4. 2025Extending AALpy with Passive Learning: A Generalized State-Merging Approachcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Early Detection of Distributed Backdoors in Multi-Agent LLM Systems: A Characterization Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability
  2. 2026Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safety
  3. 2026Insights Generator: Systematic Corpus-Level Trace Diagnostics for LLM Agents
  4. 2025ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
  5. 2026Automata from Agent Traces: Failure and Next-Step Prediction
  6. 2026MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents
  7. 2026Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents
  8. 2026REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces