When Does a Second Model Help? Cross-Model Review in LLM Verification
Not on the map. Why?
Abstract (arXiv)
Large language models now generate code, documentation, and analyses, and are increasingly used to review such output. We ask when a second review by a different model helps. Building on the author's earlier preprints, which varied context, repetition, and role structure within one model, we test model independence in a controlled experiment: 30 artifacts with 150 planted errors, 10 review conditions, and 900 review sessions with three reviewer models from two developers. In this experiment, (1) a top-tier cross-model reviewer is not significantly different in F1 from same-model review in a fresh session (CCR), which does not establish equivalence; (2) the two find partly different errors (Jaccard 41.2%); and (3) at two review calls, one CCR plus one cross-model review matches more planted errors than two CCR reviews (56.7% vs. 42.7%; Holm-adjusted p=.006), but not significantly more than two reviews by the top-tier cross-model reviewer, so model difference and reviewer capability are not separated. A lightweight cross-model reviewer scores no higher than same-model review. Withholding requirements from the reviewer raises F1 for the two lower tiers but not the top tier, in untested point estimates whose pattern depends on how failed sessions are scored. Before analysis we audited all session records, excluding one baseline run of uncertain provenance and 14 failed calls; results with all sessions are also reported. A partial check on public detector outputs from another benchmark neither replicates nor contradicts the main comparison. Records, artifacts, and scripts are available from the author on request.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
25 of 36 references.
References · 25
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
- 2023ChatDev: Communicative Agents for Software Developmentcited by 893
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
- 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
13 more
- 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
- 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 232
- 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
- 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
- 2022Automating Code Review Activities by Large-Scale Pre-trainingcited by 98
- 2024Evaluating LLMs at Detecting Errors in LLM Responsescited by 49
- 2024LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversitycited by 42
- 2024Finding Blind Spots in Evaluator LLMs with Interpretable Checklistscited by 32
- 2024Fine-Tuning and Prompt Engineering for Large Language Models-based Code Review Automationcited by 23
- 2025Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviewscited by 23
- 2026Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessionscited by 5
- 2026More Rounds, More Noise: Why Multi-Turn Review Fails to Improve Cross-Context Verificationcited by 4
- 2026Cross-Context Verification: Hierarchical Detection of Benchmark Contamination through Session-Isolated Analysiscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.