Two Calls Beat Five Agents: Evaluating Multi-Agent Pipelines Against Self-Refinement for Local Language Models
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Multi-agent LLM pipeline systems break down the task among multiple roles for better reasoning, but are benchmarked mainly with large-scale commercial models. In this study, we investigate Parishad, a structured multi-agent system involving five roles, by deploying it on Qwen2.5-7B-Instruct, a local model, on two datasets: GSM8K (500 questions) and HumanEval (164 questions), compared with prompting directly and two-call self-refinement. The multi-agent system drops GSM8K accuracy from 75.0% to 45.0% with JSON data format due to the error accumulation problem. With plaintext format, the accuracy is restored to 82.0%. A two-call self-refinement strategy (V1) can achieve 86.2% accuracy on GSM8K, with 7.4× lower token usage. However, the same V1 implementation on HumanEval---where direct accuracy is already 96.3%---actively destroys performance (66.5%). A task-aware gated redesign (V2) applied to HumanEval preserves accuracy at 95.1%. Our results demonstrate that communication format and implementation details determine outcomes more than architectural complexity, and that simpler approaches match or outperform multi-agent pipelines for local 7B model deployment. All code and data are released.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 10 of its 15 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
10 of 15 references.
Closest ancestors · 1
Same direction, newest first
Other references · 9
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023ChatDev: Communicative Agents for Software Developmentcited by 893
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration
- 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning
- 2026A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models
- 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation
- 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency
- 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study
- 2025SSR: Socratic Self-Refine for Large Language Model Reasoning
- 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration