Reviewer Capability Governs Rejection Targeting, Not Repair Skill: Evidence from LLM Execute-Review-Revise Pipelines
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Multi-agent LLM pipelines increasingly assign roles, including execution and verification, to models of different capability tiers. This is done because running a flagship model at every stage is expensive. Previous literature has established that verification stages are not always beneficial, but holds reviewer capability roughly fixed relative to the executor. We vary it. We replace the reviewer with models spanning a capability range down to one that cannot solve the problems at all, and measure the outcome of every individual rejection. This is done across a constant set of 100 olympiad mathematics problems. A cross-family mid-tier reviewer improves final accuracy by 12 percentage points, from 52 to 64 percent (p = 0.0005), with zero damaged answers. Same-model self-review attains the highest error-detection rate of any condition (0.85 recall) yet yields no significant gain: it rejects 2.1 times as often for a third the repair rate (15 against 43 percent, p = 0.0074) and falsely rejects 35 percent of its own correct answers against 2 percent for the cross-family reviewer (paired p = 0.000015). The low damage rate of self-review proves to be an artifact of revision inertia rather than reviewer quality: of 18 falsely rejected correct answers, the three where the executor complied all became wrong, while the fifteen it ignored survived unchanged. Below a capability floor the role becomes inert: our weakest reviewer changed zero of 100 final answers while doubling token cost. These findings describe a single executor-reviewer configuration on 100 problems and should be read as a controlled pilot rather than a general claim about verification stages.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 5 of its 7 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
5 of 7 references.
Closest ancestors · 1
Same direction, newest first
Other references · 4
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2026Detection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelinescited by 1
- 2026Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifiers
- 2026LLM-as-a-Judge Is Not an Oracle: Why Self-Improving Agents Need Deterministic Guardrails
- 2025Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelines
- 2026The Harness as the Only Mutable Surface: Compliance-Bounded Self-Evolution of LLM Agents in Credit Pipelines, with a Measured Admission Gate
- 2026When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
- 2026Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents
- 2026Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents
- 2026LLM-as-a-Verifier: A General-Purpose Verification Framework