# Where Does Exactly-Once Live? Model, Harness, and Tool-Contract Effects on Duplicate Side Effects in LLM Agents arXiv: https://arxiv.org/abs/2609.29095 · HTML: https://mlsanity.com/paper/2609.29095/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. - Builds on 30 of its 47 references that are in the map. - First posted on arXiv 24 Sep 2026. ## Placed in - LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/) ## Builds on (most cited first, 30 of 30) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023) - [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024) - [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023) - [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025) - [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023) - [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024) - [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024) - [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025) - [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024) - [Establishing Best Practices for Building Rigorous Agentic Benchmarks](https://mlsanity.com/paper/2507.02825/) (2025) - [Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation](https://mlsanity.com/paper/2510.11977/) (2025) - [Towards a Science of AI Agent Reliability](https://mlsanity.com/paper/2602.16666/) (2026) - [SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning](https://mlsanity.com/paper/2503.11951/) (2025) - [MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use](https://mlsanity.com/paper/2509.24002/) (2025) - [ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions](https://mlsanity.com/paper/2601.06112/) (2026) - [Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows](https://mlsanity.com/paper/2602.14849/) (2026) - [The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break](https://mlsanity.com/paper/2604.11978/) (2026) - [PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Cases](https://mlsanity.com/paper/2509.25238/) (2025) - [Cordon: Semantic Transactions for Tool-Using LLM Agents](https://mlsanity.com/paper/2606.17573/) (2026) - [When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents](https://mlsanity.com/paper/2606.05806/) (2026) - [WAREX: Web Agent Reliability Evaluation on Existing Benchmarks](https://mlsanity.com/paper/2510.03285/) (2025) - [DART: Semantic Recoverability for Structured Tool Agents](https://mlsanity.com/paper/2605.23311/) (2026) - [Hell or High Water: Evaluating Agentic Recovery from External Failures](https://mlsanity.com/paper/2508.11027/) (2025) - [Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures](https://mlsanity.com/paper/2608.02645/) (2026) - [Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems](https://mlsanity.com/paper/2609.00006/) (2026) - [AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP](https://mlsanity.com/paper/2607.11098/) (2026) - [AgentChaos: Chaos Engineering for Agent Systems via Programmatic Fault Injection](https://mlsanity.com/paper/2608.06790/) (2026) - [ToolMisuseBench: An Offline Deterministic Benchmark for Tool Misuse and Recovery in Agentic Systems](https://mlsanity.com/paper/2604.01508/) (2026) - [Agentic Transaction: Towards ACID-Compliant Agent Systems](https://mlsanity.com/paper/2608.13900/) (2026) ## Built on by (most cited first, 3 of 3) - [UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents](https://mlsanity.com/paper/2610.05622/) (2026) - [Fault-Tolerant Budget Conservation in Distributed Multi-Agent Delegation](https://mlsanity.com/paper/2610.00349/) (2026) - [RetryBench: Separating Retry Permission from Recovery Outcomes in GPT Tool Use](https://mlsanity.com/paper/2610.15884/) (2026) ## Abstract (arXiv) When a tool-using agent's write times out or returns a server error, the action may already have taken effect. Retrying blindly duplicates it -- a second charge, a second announcement, a second deployment -- while giving up skips required work. We ask where exactly-once behaviour should be enforced: in the model, in the agent harness, or in the tool contract. We introduce LIMBO, a deterministic sandbox of six services with realistic contracts (optional idempotency keys, eventually consistent and missing read paths) and twelve fault modes injected at the service boundary, including late commits, redelivery and partial batches; every episode is graded against a ledger of committed effects. Across 25,930 episodes spanning nine recent models, three production agent harnesses, two contract variants and fifteen recovery conditions, the answer depends on the fault. When an immediate read-back can reveal what happened, the model decides: frontier models instructed to act exactly once almost never duplicate a write whose acknowledgement was lost (0.5%), weaker models often do, and the model explains 53% of the explained variance. When it cannot -- the request is still in flight, or the transport delivered it twice -- the same frontier models duplicate in 56% and 74% of episodes, and the contract explains 81%. We prove that no verification-only policy is exactly-once under late commits without a bound on in-flight time. Waiting works when such a bound is short and known, but with heavy-tailed in-flight delays even an hour of waiting per episode falls short of offering an idempotency key on every write, which lowers the duplicate rate from 28% to 4% because agents use keys when they exist. The harness barely matters, a guard that attaches keys transfers across harnesses unchanged, and agents reported success in 90% of the episodes in which they had duplicated an effect.