mlsanityData to 7 Oct 2026

The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes

arXiv 2606.30653 ↗First posted 16 Jun 2026cs.CY cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification. The reliability of these pipelines depends on an implicit assumption: that the model applies relevant concepts the same way when it generates an output and later evaluates that output. We propose a new measure, generator-evaluator self-consistency, to test this assumption directly and apply it to 10 frontier models across 491 concepts. We find, first, that there is substantial variation in self-consistency. Second, we find that in a clinical setting with physician-validated mistakes (Proniakin et al., 2025), across models, those with higher self-consistency are linked to greater vulnerability to mistakes. Thus, even when models consistently apply concepts they may not be safe to deploy. This is evidence of a consistency dilemma in LLMs: self-consistency is operationally useful, but models that are more consistent are also more prone to mistakes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 73 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

36 of 73 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Self-Contradictory Reasoning Evaluation and Detection16 Nov
  2. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 34

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  9. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  10. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  11. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  12. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
22 more
  1. 2024Self-Preference Bias in LLM-as-a-Judgecited by 248
  2. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  3. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
  4. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
  5. 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
  6. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  7. 2025Great Models Think Alike and this Undermines AI Oversightcited by 53
  8. 2023Evaluating Superhuman Models with Consistency Checkscited by 47
  9. 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 43
  10. 2025Correlated Errors in Large Language Modelscited by 42
  11. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 37
  12. 2019Automatic Testing and Improvement of Machine Translationcited by 35
  13. 2019Structure-Invariant Testing for Machine Translationcited by 27
  14. 2024A Benchmark for Long-Form Medical Question Answeringcited by 25
  15. 2023MTTM: Metamorphic Testing for Textual Content Moderation Softwarecited by 20
  16. 2023METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualitiescited by 20
  17. 2020Testing Machine Translation via Referential Transparencycited by 19
  18. 2025FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Modelscited by 18
  19. 2025Potemkin Understanding in Large Language Modelscited by 14
  20. 2025Metamorphic Testing of Large Language Models for Natural Language Processingcited by 12
  21. 2024An Empirical Study on Large Language Models in Accuracy and Robustness under Chinese Industrial Scenarioscited by 3
  22. 2025Automatic Replication of LLM Mistakes in Medical Conversationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Existing LLMs Are Not Self-Consistent For Simple Tasks
  2. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  3. 2024Evaluating the Consistency of LLM Evaluators
  4. 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey
  5. 2023Self-Consistency of Large Language Models under Ambiguity
  6. 2023Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs
  7. 2026Position: It's Time to Optimize LLMs for Self-Consistency
  8. 2024Evaluating Consistency and Reasoning Capabilities of Large Language Models