mlsanityData to 14 Oct 2026

Review Rarely Beats the Stronger Model: Separating Verification from Capability Transfer in LLM Pipelines

arXiv 2610.15897 ↗First posted 11 Oct 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Qwen3-8B reviewing Llama-3.1-8B in math adds 33.2 points, yet Qwen3-8B alone is 6.4 points more accurate than the result. Reviewing another model's answer is usually judged by its gain over the generator. We show that this confuses verification, which adds accuracy beyond either model alone, with capability transfer, where a stronger reviewer supplies answers it could have given on its own. Measured against the stronger model alone, across 48 pairs of 12 open-weight models (3B to 72B) in math, knowledge, code and reasoning, review beats it in only 9 of 96 review directions, and does worse the larger the capability gap. The reason is anchoring: the reviewer mostly follows the generator, discarding its own correct answers. Review costs 2.6 to 2.8 times as much as the stronger model alone, while a selector using the same two calls is 2.7 to 4.2 points more accurate than review and cheaper. Studies of verification should report the stronger model alone, not only the gain over the generator.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 58 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

32 of 58 references.

References · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  10. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  11. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  12. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
20 more
  1. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  2. 2022Generating Sequences by Learning to Self-Correctcited by 277
  3. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  4. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 217
  5. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  6. 2023Large Language Model Routing with Benchmark Datasetscited by 172
  7. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifierscited by 62
  8. 2025Seed-Coder: Let the Code Model Curate Data for Itselfcited by 49
  9. 2025When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoningcited by 42
  10. 2024Conformity in Large Language Modelscited by 34
  11. 2026JEV-as-a-Judge: Accept When Confident, Escalate When Unsurecited by 30
  12. 2025When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoningcited by 21
  13. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 15
  14. 2026More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judgescited by 10
  15. 2026Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformitycited by 8
  16. 2026Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systemscited by 6
  17. 2026Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?cited by 4
  18. 2026When to Screen, When to Bypass: LLM-Judges in Resource-Scarce AI-Human Workflowcited by 2
  19. 2026Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoningcited by 2
  20. 2026Commit-first LLM judging inherits the judge's own errorscited by 2

Built on by

Not cited yet.