# Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency

arXiv: https://arxiv.org/abs/2309.17272 · HTML: https://mlsanity.com/paper/2309.17272/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [Natural Language to Code Translation with Execution](https://mlsanity.com/paper/2204.11454/) (2022)
- [Coder Reviewer Reranking for Code Generation](https://mlsanity.com/paper/2211.16490/) (2022)
- [Solving Math Word Problems via Cooperative Reasoning induced Language Models](https://mlsanity.com/paper/2210.16257/) (2022)
- [Recitation-Augmented Language Models](https://mlsanity.com/paper/2210.01296/) (2022)
- [ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers](https://mlsanity.com/paper/2305.14591/) (2023)
- [Agile Software Development Methods: Review and Analysis](https://mlsanity.com/paper/1709.08439/) (2017)

## Built on by (most cited first, 30 of 46)

- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step](https://mlsanity.com/paper/2402.16906/) (2024)
- [KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding](https://mlsanity.com/paper/2503.02951/) (2025)
- [S*: Test Time Scaling for Code Generation](https://mlsanity.com/paper/2502.14382/) (2025)
- [ACECODER: Acing Coder RL via Automated Test-Case Synthesis](https://mlsanity.com/paper/2502.01718/) (2025)
- [Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives](https://mlsanity.com/paper/2401.02009/) (2024)
- [Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning](https://mlsanity.com/paper/2506.03136/) (2025)
- [rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset](https://mlsanity.com/paper/2505.21297/) (2025)
- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [Automated Proof Generation for Rust Code via Self-Evolution](https://mlsanity.com/paper/2410.15756/) (2024)
- [RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System](https://mlsanity.com/paper/2602.02488/) (2026)
- [EVOR: Evolving Retrieval for Code Generation](https://mlsanity.com/paper/2402.12317/) (2024)
- [Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation](https://mlsanity.com/paper/2407.02056/) (2024)
- [The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?](https://mlsanity.com/paper/2402.19475/) (2024)
- [In Line with Context: Repository-Level Code Generation via Context Inlining](https://mlsanity.com/paper/2601.00376/) (2026)
- [Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs](https://mlsanity.com/paper/2406.11514/) (2024)
- [OR-R1: Automating Modeling and Solving of Operations Research Optimization Problem via Test-Time Reinforcement Learning](https://mlsanity.com/paper/2511.09092/) (2025)
- [Scaling LLM Inference with Optimized Sample Compute Allocation](https://mlsanity.com/paper/2410.22480/) (2024)
- [Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models](https://mlsanity.com/paper/2407.00456/) (2024)
- [AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptation](https://mlsanity.com/paper/2503.19693/) (2025)
- [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning](https://mlsanity.com/paper/2501.11877/) (2025)
- [LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness](https://mlsanity.com/paper/2502.18489/) (2025)
- [ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models](https://mlsanity.com/paper/2408.00994/) (2024)
- [Integrative Decoding: Improve Factuality via Implicit Self-consistency](https://mlsanity.com/paper/2410.01556/) (2024)
- [Generative AI in Transportation Planning: A Survey](https://mlsanity.com/paper/2503.07158/) (2025)
- [Graph of Verification: Structured Verification of LLM Reasoning with Directed Acyclic Graphs](https://mlsanity.com/paper/2506.12509/) (2025)
- [Scaling Agentic Verifier for Competitive Coding](https://mlsanity.com/paper/2602.04254/) (2026)
- [Flaming-hot Initiation with Regular Execution Sampling for Large Language Models](https://mlsanity.com/paper/2410.21236/) (2024)

## Abstract (arXiv)

Large language models (LLMs) have exhibited remarkable ability in code generation. However, generating the correct solution in a single attempt still remains a challenge. Prior works utilize verification properties in software engineering to verify and re-rank solutions in a majority voting manner. But the assumption behind them that generated verification properties have better qualities than solutions may not always hold. In this paper, we treat them equally as different perspectives of LLMs' reasoning processes. We propose the Multi-Perspective Self-Consistency (MPSC) framework incorporating both inter- and intra-consistency across outputs from multiple perspectives. Specifically, we prompt LLMs to generate diverse outputs from three perspectives, Solution, Specification and Test case, constructing a 3-partite graph. With two measure functions of consistency, we embed both inter- and intra-consistency information into the graph. The optimal choice of solutions is then determined based on analysis in the graph. MPSC significantly boosts performance of foundation models (ChatGPT in this paper) on various benchmarks, including HumanEval (+15.91%), MBPP (+6.43%) and CodeContests (+9.37%), even surpassing GPT-4.