Deterministic Replay for AI Agent Systems
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
AI agent systems that couple large language models (LLMs) with external tools and APIs are inherently non-deterministic: LLM sampling variance, external API state, CDN infrastructure headers, and execution-environment noise collectively prevent any prior agent run from being faithfully re-executed. Existing observability platforms capture execution logs but cannot reproduce a run in isolation. We present agrepl, a developer-first CLI framework for deterministic replay of agent executions. agrepl intercepts all external interactions at the transport layer via a man-in-the-middle (MITM) proxy, serialises them as structured execution traces, and replays them in a strictly isolated environment with zero outbound network access. We formalise the agent execution model, define the request-key matching function K(s), and prove the determinism invariant. We introduce a noise-aware diff algorithm classifying HTTP header divergence into signal and noise tiers. Empirical evaluation across five workloads (n = 250 replay instances) demonstrates replay fidelity F = 1.0 and a median per-step latency reduction of 98.3%. agrepl is implemented in Go, ships as a single static binary, and is released under the MIT licence. Keywords: AI agents, deterministic replay, LLM debugging, reproducibility, MITM proxy, execution tracing, record/replay systems.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 2 of its 20 references that are in the map. First posted on arXiv 30 Apr 2026.
Builds on
2 of 20 references.
References · 2
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Talking About Large Language Modelscited by 174
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Get Experience from Practice: LLM Agents with Record & Replay
- 2026Localizing Emergent Failures in Agentic AI: Recovering Minimal Repair Families via Counterfactual Replay
- 2026Knowledge-Based Zero-Replay Debugging of Multi-Agent LLM Traces
- 2026The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong World
- 2026Revisable by Design: A Theory of Streaming LLM Agent Execution
- 2026Observability for Delegated Execution in Agentic AI Systems
- 2026AgentTrace: A Structured Logging Framework for Agent System Observability
- 2025Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems