Semantic Early-Stopping for Iterative LLM Agent Loops
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Multi-agent large language model (LLM) loops, for example a Writer that drafts and a Critic that revises, are almost always terminated by a fixed iteration cap (max_iterations). This is a syntactic kill-switch: it is blind to whether the answer is still improving, so it over-spends tokens on easy inputs and truncates hard ones. We study semantic early-stopping: the loop halts when consecutive draft embeddings stop changing in meaning (cosine distance with a patience window) and the answer's measured quality stops improving. Our work makes three contributions. First, an honest theoretical footing: we prove deterministic termination and well-definedness and machine-check these claims, while treating the convergence of the distance sequence as an empirically tested conjecture rather than a (previously over-claimed) Banach contraction. Second, a judge-efficient evaluation protocol: we generate each question's full trajectory once, replay every stopping policy over the identical drafts, and cache every LLM-judge call, yielding a strictly paired efficiency-versus-quality comparison at low cost; we further separate operational tokens (charged to a policy) from evaluation tokens (a measurement instrument). Third, an empirical study on multi-hop retrieval-augmented question answering (HotpotQA). On the 60-question test split, a judge-free semantic stopper reduces operational tokens by 38% relative to max_iterations at parity quality (Delta-IS = -0.004, p = 0.81), whereas the full quality-gated variant is counter-productive because its per-round judging dominates cost. An oracle that selects the best round attains +0.115 Information Score over every practical policy (p ~ 4e-11), reframing the problem from "when to stop" (easy) to "which round is best" (open).
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 5 of its 12 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
5 of 12 references.
Closest ancestors · 2
Same direction, newest first
- 2026Phase-Scheduled Multi-Agent Systems for Token-Efficient Coordination19 Apr
- 2026AdaptOrch: Task-Adaptive Multi-Agent Orchestration in the Era of LLM Performance Convergence18 Feb
Other references · 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
- 2025Early Stopping Chain-of-thoughts in Large Language Models
- 2026Monitoring latent reasoning in looped language models
- 2025seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs
- 2026A Mechanistic Analysis of Looped Reasoning Language Models
- 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
- 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
- 2026Adaptive Stopping for Multi-Turn LLM Reasoning