mlsanityData to 14 Oct 2026

Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers

arXiv 2502.20379 ↗First posted 27 Feb 2025cs.AIcited by 62

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

By utilizing more computational resources at test-time, large language models (LLMs) can improve without additional training. One common strategy uses verifiers to evaluate candidate outputs. In this work, we propose a novel scaling dimension for test-time compute: scaling the number of verifiers. We introduce Multi-Agent Verification (MAV) as a test-time compute paradigm that combines multiple verifiers to improve performance. We propose using Aspect Verifiers (AVs), off-the-shelf LLMs prompted to verify different aspects of outputs, as one possible choice for the verifiers in a MAV system. AVs are a convenient building block for MAV since they can be easily combined without additional training. Moreover, we introduce BoN-MAV, a simple multi-agent verification algorithm that combines best-of-n sampling with multiple verifiers. BoN-MAV demonstrates stronger scaling patterns than self-consistency and reward model verification, and we demonstrate both weak-to-strong generalization, where combining weak verifiers improves even stronger LLMs, and self-improvement, where the same base model is used to both generate and verify outputs. Our results establish scaling the number of verifiers as a promising new dimension for improving language model performance at test-time.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 79 of its 86 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

79 of 86 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct
  3. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  5. 2023Large Language Model Guided Tree-of-Thought15 May
  6. 2023Self-Evaluation Guided Beam Search for Reasoning1 May
  7. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  8. 2022Large Language Models Can Self-Improve20 Oct

Other references · 71

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
38 more
  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2023Mistral 7Bcited by 3,164
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  10. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  12. 2020Learning to summarize from human feedbackcited by 2,588
  13. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  14. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  15. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  16. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  17. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  18. 2016Concrete Problems in AI Safetycited by 1,642
  19. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  20. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  21. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  22. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  23. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
  24. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  25. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  26. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  27. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  28. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  29. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  30. 2022PAL: Program-aided Language Modelscited by 652
  31. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  32. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  33. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
  34. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
  35. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  36. 2018AI safety via debatecited by 400
  37. 2022Self-critiquing models for assisting human evaluatorscited by 380
  38. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 37.

62 papers.

Sort:

By area:

  1. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimizationcited by 56
  2. 2025LLM Collaboration With Multi-Agent Reinforcement Learningcited by 49
  3. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 43
  4. 2025Video-T1: Test-Time Scaling for Video Generationcited by 40
  5. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 24
  6. 2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Mathcited by 22
  7. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Computecited by 20
  8. 2025Variation in Verification: Understanding Verification Dynamics in Large Language Modelscited by 19
  9. 2026World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetrycited by 18
  10. 2026Agent-as-a-Judgecited by 17
  11. 2025LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perceptioncited by 16
  12. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
38 more
  1. 2025SSRL: Self-Search Reinforcement Learningcited by 14
  2. 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoningcited by 14
  3. 2025The Art of Scaling Test-Time Compute for Large Language Modelscited by 12
  4. 2025Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelinescited by 10
  5. 2026Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantificationcited by 10
  6. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 9
  7. 2025Scaling Generative Verifiers For Natural Language Mathematical Proof Verification And Selectioncited by 9
  8. 2026Tool Verification for Test-Time Reinforcement Learningcited by 9
  9. 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Criticcited by 8
  10. 2025CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoringcited by 7
  11. 2025Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scalingcited by 7
  12. 2025Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Explorationcited by 7
  13. 2025Slm-mux: Orchestrating small language models for reasoningcited by 6
  14. 2026Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertaintycited by 6
  15. 2025Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scalingcited by 5
  16. 2026ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inferencecited by 5
  17. 2026Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verificationcited by 5
  18. 2026Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactionscited by 5
  19. 2026FUSE: Ensembling Verifiers with Zero Labeled Datacited by 4
  20. 2026IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agentscited by 4
  21. 2026On the Generalization Gap in Self-Evolving Language Model Reasoningcited by 4
  22. 2025Less Data Less Tokens: Multilingual Unification Learning for Efficient Test-Time Reasoning in LLMscited by 3
  23. 2025Communication and Verification in LLM Agents towards Collaboration under Information Asymmetrycited by 3
  24. 2025EVE: A Generator-Verifier System for Generative Policiescited by 3
  25. 2026Locating Hidden Failures Makes Long-Horizon Agents More Reliablecited by 3
  26. 2026Interactive Critique-Revision Training for Reliable Structured LLM Generationcited by 2
  27. 2026Agentic Systems as Boosting Weak Reasoning Modelscited by 2
  28. 2026Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaborationcited by 2
  29. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategiescited by 1
  30. 2026One-Token Verification for Reasoning Correctness Estimationcited by 1
  31. 2026Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Modelscited by 1
  32. 2026Efficient Agentic Reasoning Through Self-Regulated Simulative Planningcited by 1
  33. 2026Granularity-Regulated Adaptive Computational Efficiency for Optimal Verification in Test-Time Scalingcited by 1
  34. 2026Engineering Reliable Commit Gates for Agentic AI: Cost-Aware Verification Portfolios under Common-Mode Data Failurescited by 1
  35. 2026FluxLite: Inference-Time Proposal Control for Discrete Diffusion Modelscited by 1
  36. 2025FOR-Prompting: From Objection to Revision via an Asymmetric Prompting Protocolcited by 0
  37. 2026A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigmscited by 0
  38. 2026A Single Revision Step Improves Token-Efficient LLM Reasoningcited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Test-time Compute for LLM Agents
  2. 2026Parallel Test-Time Scaling with Multi-Sequence Verifiers
  3. 2025Trust but Verify! A Survey on Verification Design for Test-time Scaling
  4. 2026LLM-as-a-Verifier: A General-Purpose Verification Framework
  5. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Tasks
  6. 2026Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling
  7. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results
  8. 2026Scaling Test-Time Compute for Agentic Coding