mlsanityData to 14 Oct 2026

Universal Self-Consistency for Large Language Model Generation

arXiv 2311.17311 ↗First posted 29 Nov 2023cs.CL cs.AIcited by 203

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-consistency with chain-of-thought prompting (CoT) has demonstrated remarkable performance gains on various challenging tasks, by utilizing multiple reasoning paths sampled from large language models (LLMs). However, self-consistency relies on the answer extraction process to aggregate multiple solutions, which is not applicable to free-form answers. In this work, we propose Universal Self-Consistency (USC), which leverages LLMs themselves to select the most consistent answer among multiple candidates. We evaluate USC on a variety of benchmarks, including mathematical reasoning, code generation, long-context summarization, and open-ended question answering. On open-ended generation tasks where the original self-consistency method is not applicable, USC effectively utilizes multiple samples and improves the performance. For mathematical reasoning, USC matches the standard self-consistency performance without requiring the answer formats to be similar. Finally, without access to execution results, USC also matches the execution-based voting performance on code generation.

Key facts. Cited by 203 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 42 of its 47 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

42 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing19 May

Other references · 40

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  12. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
28 more
  1. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  2. 2023PaLM 2 Technical Reportcited by 1,162
  3. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  4. 2023Large Language Models as Optimizerscited by 864
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  6. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  7. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  8. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  9. 2022CodeT: Code Generation with Generated Testscited by 384
  10. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  11. 2021Efficient Attentions for Long Document Summarizationcited by 345
  12. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  13. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 239
  14. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
  15. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 176
  16. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  17. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  18. 2022Natural Language to Code Translation with Executioncited by 124
  19. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 94
  20. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 87
  21. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  22. 2022Coder Reviewer Reranking for Code Generationcited by 84
  23. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  24. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 79
  25. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 51
  26. 2023Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistencycited by 46
  27. 2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChaincited by 31
  28. 2022N-Best Hypotheses Reranking for Text-To-SQL Systemscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 42; 2025: 81; 2026: 78.

203 papers.

Sort:

By area:

  1. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  2. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  3. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 187
  4. 2024Distilling System 2 into System 1cited by 159
  5. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  6. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 134
  7. 2024To Believe or Not to Believe Your LLMcited by 120
  8. 2024Approaching Human-Level Forecasting with Language Modelscited by 107
  9. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 76
  10. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 72
  11. 2025SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scalingcited by 54
  12. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 54
38 more
  1. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  2. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  3. 2024Current state of LLM Risks and AI Guardrailscited by 46
  4. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  5. 2025When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoningcited by 42
  6. 2025Video-T1: Test-Time Scaling for Video Generationcited by 40
  7. 2024Calibrating Long-form Generations from Large Language Modelscited by 32
  8. 2023Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Usecited by 27
  9. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
  10. 2024Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberationcited by 26
  11. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 26
  12. 2024Soft Self-Consistency Improves Language Model Agentscited by 25
  13. 2024LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMscited by 25
  14. 2024Watermarking Techniques for Large Language Models: A Surveycited by 23
  15. 2024A Survey on the Honesty of Large Language Modelscited by 23
  16. 2024In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translationcited by 22
  17. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generationcited by 21
  18. 2024The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?cited by 20
  19. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Computecited by 20
  20. 2024Experts Don't Cheat: Learning What You Don't Know By Predicting Pairscited by 19
  21. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  22. 2024Long-Form Answers to Visual Questions from Blind and Low Vision Peoplecited by 18
  23. 2025Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoningcited by 18
  24. 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Modelscited by 18
  25. 2026Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judgecited by 18
  26. 2024Tapilot-Crossing: Benchmarking and Evolving LLMs Towards Interactive Data Analysis Agentscited by 17
  27. 2025OR-R1: Automating Modeling and Solving of Operations Research Optimization Problem via Test-Time Reinforcement Learningcited by 17
  28. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  29. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  30. 2024Understanding the Effects of Iterative Prompting on Truthfulnesscited by 14
  31. 2024Multi-expert Prompting Improves Reliability, Safety, and Usefulness of Large Language Modelscited by 14
  32. 2025Conformal Language Model Reasoning with Coherent Factualitycited by 14
  33. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoningcited by 14
  34. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitationcited by 13
  35. 2025Enhancing Mathematical Reasoning in Large Language Models with Self-Consistency-Based Hallucination Detectioncited by 13
  36. 2024When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Modelscited by 12
  37. 2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMscited by 12
  38. 2025Learning from Diverse Reasoning Paths with Routing and Collaborationcited by 12

50 most cited of 203 All 203 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
  2. 2025Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning
  3. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling
  4. 2026Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
  5. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
  6. 2026Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs
  7. 2023Self-Consistency of Large Language Models under Ambiguity
  8. 2026Strategic Self-Consistency