mlsanityData to 7 Oct 2026

Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning

arXiv 2607.15388 ↗First posted 16 Jul 2026cs.AIcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Many math- and science-oriented agent systems use hierarchical designs with specialized reviewer roles, assuming that a dedicated review stage should help turn wrong candidates into correct ones. We test this assumption on 4,181 verifier-grounded Omni-MATH problems using matched gpt-oss-120b actors. Collaboration adds little on the easiest tiers, but from tier 4 onward the gains open sharply; in this harder regime, broadcast-style peer discussion reaches higher final accuracy than a planner-executor-reviewer pipeline (PER). We ask whether this gap is explained by reviewer quality or by whether critique changes the next answer the protocol carries forward. It is not explained by reviewer precision alone: PER's reviewer is more precise than broadcast's (0.861 vs. 0.644), yet evaluator-verified useful critique is much less likely to change the next candidate and produces lower reviewer-guided repair. These results show that reviewer detection quality and critique uptake are empirically separable. Within matched PER interventions, forcing explicit acknowledgment lowers final accuracy, while embedding reviewer guidance directly in the solver's working context partially improves follow-through without closing the gap. Overall, reviewer-centric evaluation can overstate system quality: a protocol may spot errors well yet still fail to solve more problems if it does not act on those critiques.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 31 of its 38 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

31 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Towards a Science of Scaling Agent Systems9 Dec
  2. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning11 Nov
  3. 2025Reinforce LLM Reasoning through Multi-Agent Reflection10 Jun
  4. 2024MALT: Improving Reasoning with Multi-Agent LLM Training2 Dec

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  9. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  10. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  11. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  12. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
15 more
  1. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 277
  2. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  3. 2024AI Agents That Mattercited by 199
  4. 2025Survey on Evaluation of LLM-based Agentscited by 190
  5. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  6. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  7. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 94
  8. 2025xVerify: Efficient Answer Verifier for Reasoning Model Evaluationscited by 53
  9. 2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluatorscited by 42
  10. 2024Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMscited by 28
  11. 2025Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMscited by 12
  12. 2023On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answeringcited by 10
  13. 2026Benchmarks Saturate When The Model Gets Smarter Than The Judgecited by 10
  14. 2025Decomposing LLM Self-Correction: The Accuracy-Correction Paradox and Error Depth Hypothesiscited by 8
  15. 2026How Many Tries Does It Take? Iterative Self-Repair in LLM Code Generation Across Model Scales and Benchmarkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Reviewer Capability Governs Rejection Targeting, Not Repair Skill: Evidence from LLM Execute-Review-Revise Pipelinescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness
  2. 2026RMA: an Agentic System for Research-Level Mathematical Problems
  3. 2025Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning
  4. 2026ResearchMath-14K: Scaling Research-Level Mathematics via Agents
  5. 2026ProofCouncil: An LLM Agent for Solving Open Mathematical Problems
  6. 2026When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning
  7. 2026The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
  8. 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration