mlsanityData to 14 Oct 2026

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency

arXiv 2309.17272 ↗First posted 29 Sep 2023cs.CL cs.AI cs.SEcited by 46

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large language models (LLMs) have exhibited remarkable ability in code generation. However, generating the correct solution in a single attempt still remains a challenge. Prior works utilize verification properties in software engineering to verify and re-rank solutions in a majority voting manner. But the assumption behind them that generated verification properties have better qualities than solutions may not always hold. In this paper, we treat them equally as different perspectives of LLMs' reasoning processes. We propose the Multi-Perspective Self-Consistency (MPSC) framework incorporating both inter- and intra-consistency across outputs from multiple perspectives. Specifically, we prompt LLMs to generate diverse outputs from three perspectives, Solution, Specification and Test case, constructing a 3-partite graph. With two measure functions of consistency, we embed both inter- and intra-consistency information into the graph. The optimal choice of solutions is then determined based on analysis in the graph. MPSC significantly boosts performance of foundation models (ChatGPT in this paper) on various benchmarks, including HumanEval (+15.91%), MBPP (+6.43%) and CodeContests (+9.37%), even surpassing GPT-4.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 29 of its 44 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

29 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Teaching Large Language Models to Self-Debug11 Apr
  2. 2023LEVER: Learning to Verify Language-to-Code Generation with Execution16 Feb
  3. 2022Competition-Level Code Generation with AlphaCode8 Feb

Other references · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
14 more
  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  3. 2023StarCoder: may the source be with you!cited by 799
  4. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  5. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  6. 2022CodeT: Code Generation with Generated Testscited by 384
  7. 2023Self-collaboration Code Generation via ChatGPTcited by 258
  8. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  9. 2022Natural Language to Code Translation with Executioncited by 124
  10. 2022Coder Reviewer Reranking for Code Generationcited by 84
  11. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Modelscited by 77
  12. 2022Recitation-Augmented Language Modelscited by 65
  13. 2023ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifierscited by 41
  14. 2017Agile Software Development Methods: Review and Analysiscited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 17; 2025: 16; 2026: 12.

46 papers.

Sort:

By area:

  1. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  2. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  3. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 119
  4. 2025S*: Test Time Scaling for Code Generationcited by 98
  5. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 93
  6. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectivescited by 71
  7. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 56
  8. 2025rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Datasetcited by 51
  9. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  10. 2024Automated Proof Generation for Rust Code via Self-Evolutioncited by 33
  11. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 24
  12. 2024EVOR: Evolving Retrieval for Code Generationcited by 22
34 more
  1. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generationcited by 21
  2. 2024The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?cited by 20
  3. 2026In Line with Context: Repository-Level Code Generation via Context Inliningcited by 18
  4. 2024Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMscited by 17
  5. 2025OR-R1: Automating Modeling and Solving of Operations Research Optimization Problem via Test-Time Reinforcement Learningcited by 17
  6. 2024Scaling LLM Inference with Optimized Sample Compute Allocationcited by 16
  7. 2024Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Modelscited by 15
  8. 2025AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptationcited by 15
  9. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  10. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  11. 2025From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuningcited by 12
  12. 2025LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctnesscited by 10
  13. 2024ArchCode: Incorporating Software Requirements in Code Generation with Large Language Modelscited by 8
  14. 2024Integrative Decoding: Improve Factuality via Implicit Self-consistencycited by 8
  15. 2025Generative AI in Transportation Planning: A Surveycited by 8
  16. 2025Graph of Verification: Structured Verification of LLM Reasoning with Directed Acyclic Graphscited by 8
  17. 2026Scaling Agentic Verifier for Competitive Codingcited by 7
  18. 2024Flaming-hot Initiation with Regular Execution Sampling for Large Language Modelscited by 6
  19. 2024Automated Prompt Engineering for Cost-Effective Code Generation Using Evolutionary Algorithmcited by 5
  20. 2026EvolveCoder: Evolving Test Cases via Adversarial Verification for Code Reinforcement Learningcited by 5
  21. 2024DOCE: Finding the Sweet Spot for Execution-Based Code Generationcited by 4
  22. 2025When Dimensionality Hurts: The Role of LLM Embedding Compression for Noisy Regression Taskscited by 2
  23. 2025AgentTutor: Empowering Personalized Learning with Multi-Turn Interactive Teaching in Intelligent Education Systemscited by 2
  24. 2026ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code Generationcited by 2
  25. 2024PEDAL: Enhancing Greedy Decoding with Large Language Models using Diverse Exemplarscited by 1
  26. 2024Assured Automatic Programming via Large Language Modelscited by 1
  27. 2025Towards Functional Correctness of Large Code Models with Selective Generationcited by 1
  28. 2026CascadeMind at SemEval-2026 Task 4: A Hybrid Neuro-Symbolic Cascade for Narrative Similaritycited by 1
  29. 2026MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learningcited by 1
  30. 2026DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocodecited by 0
  31. 2026Standing on the Shoulders of Giants: Stabilized Knowledge Distillation for Cross--Language Code Clone Detectioncited by 0
  32. 2026XSearch: Explainable Code Search via Concept-to-Code Alignmentcited by 0
  33. 2026Information-Gain Rewards over Diversity-Pruned Tests: GT-Anchored Verifier Co-Training for Reliable Code Generationcited by 0
  34. 2026Semantic Abstraction for Natural Language Inference: a Methodological Framework for Discovering and Compensating Semantic Knowledge and Reasoning Gaps in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain
  2. 2026Inferring Code Correctness from Specification
  3. 2026ExeCRE: Execution-Consistency Guided Reliability Estimation for Self-Correcting Code Generation
  4. 2025Towards Advancing Code Generation with Large Language Models: A Research Roadmap
  5. 2023Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs
  6. 2025Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning
  7. 2024CodeMind: Evaluating Large Language Models for Code Reasoning
  8. 2025Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs