mlsanityData to 7 Oct 2026

Deterministic Replay for AI Agent Systems

arXiv 2607.16200 ↗First posted 30 Apr 2026cs.AIcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

AI agent systems that couple large language models (LLMs) with external tools and APIs are inherently non-deterministic: LLM sampling variance, external API state, CDN infrastructure headers, and execution-environment noise collectively prevent any prior agent run from being faithfully re-executed. Existing observability platforms capture execution logs but cannot reproduce a run in isolation. We present agrepl, a developer-first CLI framework for deterministic replay of agent executions. agrepl intercepts all external interactions at the transport layer via a man-in-the-middle (MITM) proxy, serialises them as structured execution traces, and replays them in a strictly isolated environment with zero outbound network access. We formalise the agent execution model, define the request-key matching function K(s), and prove the determinism invariant. We introduce a noise-aware diff algorithm classifying HTTP header divergence into signal and noise tiers. Empirical evaluation across five workloads (n = 250 replay instances) demonstrates replay fidelity F = 1.0 and a median per-step latency reduction of 98.3%. agrepl is implemented in Go, ships as a single static binary, and is released under the MIT licence. Keywords: AI agents, deterministic replay, LLM debugging, reproducibility, MITM proxy, execution tracing, record/replay systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 2 of its 20 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

2 of 20 references.

References · 2

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Talking About Large Language Modelscited by 174

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Get Experience from Practice: LLM Agents with Record & Replay
  2. 2026Localizing Emergent Failures in Agentic AI: Recovering Minimal Repair Families via Counterfactual Replay
  3. 2026Knowledge-Based Zero-Replay Debugging of Multi-Agent LLM Traces
  4. 2026The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
  5. 2026Revisable by Design: A Theory of Streaming LLM Agent Execution
  6. 2026Observability for Delegated Execution in Agentic AI Systems
  7. 2026AgentTrace: A Structured Logging Framework for Agent System Observability
  8. 2025Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems