mlsanityData to 7 Oct 2026

Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning

arXiv 2504.09772 ↗First posted 14 Apr 2025cs.AIcited by 29

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Test-Time Scaling has emerged as a powerful method to extend the reasoning capabilities of Large Language Models. However, single-agent TTS faces significant scalability bottlenecks, as excessively long reasoning traces lead to increased inference costs and stability issues caused by context management failures. To address these limitations, we propose leveraging Multi-Agent Systems as a structural upgrade to standard TTS. By decomposing monolithic reasoning chains into distinct, manageable contexts across multiple agents, MAS offers a more robust framework for scaling reasoning. We validate this approach by introducing M500, a dataset comprising 500 high-quality multi-agent, multi-turn collaborative reasoning traces generated via DeepSeek-R1. Through Supervised Fine-Tuning on M500, we enable open-source models to internalize collaborative reasoning patterns and show improved TTS performance in MAS. Furthermore, we propose an adaptive scaling strategy incorporating a ``CEO'' agent to dynamically guide the reasoning process and optimize collaboration depth. Extensive experiments within the AgentVerse framework demonstrate that our fine-tuned models, Qwen2.5-32B-MAS and Phi4-14B-MAS, significantly outperform their base counterparts. Codes are available at https://github.com/jincan333/MAS-TTS.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 84 of its 111 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

84 of 111 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024MALT: Improving Reasoning with Multi-Agent LLM Training2 Dec
  2. 2024Qwen2.5-Coder Technical Report18 Sep
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models6 Oct

Other references · 81

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
38 more
  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2024Qwen2 Technical Reportcited by 2,076
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  12. 2025s1: Simple test-time scalingcited by 1,311
  13. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  14. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  15. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  16. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  17. 2022In-context Learning and Induction Headscited by 930
  18. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  19. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  20. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  21. 2022Large Language Models Can Self-Improvecited by 656
  22. 2022PAL: Program-aided Language Modelscited by 635
  23. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  24. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  25. 2025LIMO: Less is More for Reasoningcited by 488
  26. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  27. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  28. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
  29. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  30. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  31. 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 225
  32. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  33. 2023Planning with Large Language Models for Code Generationcited by 179
  34. 2025Multi-agent Architecture Search via Agentic Supernetcited by 164
  35. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  36. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
  37. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  38. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 9.

29 papers.

Sort:

By area:

  1. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  2. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  3. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  4. 2025AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learningcited by 13
  5. 2025MTL-UE: Learning to Learn Nothing for Multi-Task Learningcited by 11
  6. 2025Scaling over Scaling: Exploring Test-Time Scaling Plateau in Large Reasoning Modelscited by 11
  7. 2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMscited by 11
  8. 2025M^3-Bench: Multi-Modal, Multi-Hop, Multi-Threaded Tool-Using MLLM Agent Benchmarkcited by 11
  9. 2026Behavior Knowledge Merge in Reinforced Agentic Modelscited by 10
  10. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  11. 2025Web Fraud Attacks Against LLM-Driven Multi-Agent Systemscited by 8
  12. 2026Tool Verification for Test-Time Reinforcement Learningcited by 6
17 more
  1. 2026Agent Skill Evaluation and Evolution: Frameworks and Benchmarkscited by 6
  2. 2025CoNav: Collaborative Cross-Modal Reasoning for Embodied Navigationcited by 5
  3. 2025A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Searchcited by 5
  4. 2026MAS-ProVe: Understanding the Process Verification of Multi-Agent Systemscited by 5
  5. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Modelscited by 5
  6. 2025LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOScited by 4
  7. 2025A Survey of Deep Learning for Geometry Problem Solvingcited by 4
  8. 2025Language-Driven Coordination and Learning in Multi-Agent Simulation Environmentscited by 3
  9. 2026Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safetycited by 3
  10. 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generationcited by 3
  11. 2025Structured Debate Improves Corporate Credit Reasoning in Financial AIcited by 1
  12. 2025Explainable reinforcement learning from human feedback to improve alignmentcited by 1
  13. 2025E-bike agents: Large Language Model-Driven E-Bike Accident Analysis and Severity Predictioncited by 0
  14. 2025TokenBlowUp: Resolving Representational Singularities in LLM Token Spaces via Monoidal Transformationscited by 0
  15. 2025Scaling Unverifiable Rewards: A Case Study on Visual Insightscited by 0
  16. 2026DIANOIA: Diagnostic Decomposition and Joint Optimization for Multi-Agent Reasoningcited by 0
  17. 2026Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoffcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling
  2. 2026TMAS: Scaling Test-Time Compute via Multi-Agent Synergy
  3. 2025Adaptive Coopetition: Leveraging Coarse Verifier Signals for Resilient Multi-Agent LLM Reasoning
  4. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness
  5. 2026AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents
  6. 2026Orchestrating Intelligence: Confidence-Aware Routing for Efficient Multi-Agent Collaboration across Multi-Scale Models
  7. 2025A Survey of Scaling in Large Language Model Reasoning
  8. 2024Scaling Large Language Model-based Multi-Agent Collaboration