# Ranked Voting based Self-Consistency of Large Language Models

arXiv: https://arxiv.org/abs/2505.10772 · HTML: https://mlsanity.com/paper/2505.10772/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 16 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [Cumulative Reasoning with Large Language Models](https://mlsanity.com/paper/2308.04371/) (2023)
- [Improving Your Model Ranking on Chatbot Arena by Vote Rigging](https://mlsanity.com/paper/2501.17858/) (2025)

## Built on by (most cited first, 19 of 19)

- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [Optimal Self-Consistency for Efficient Reasoning with Large Language Models](https://mlsanity.com/paper/2511.12309/) (2025)
- [Agentic Test-Time Scaling for WebAgents](https://mlsanity.com/paper/2602.12276/) (2026)
- [Beyond Consensus: Trace-Level Synthesis in Mixture of Agents](https://mlsanity.com/paper/2605.29116/) (2026)
- [DEL-ToM: Inference-Time Scaling for Theory-of-Mind Reasoning via Dynamic Epistemic Logic](https://mlsanity.com/paper/2505.17348/) (2025)
- [Beyond Majority Voting: Efficient Best-Of-N with Radial Consensus Score](https://mlsanity.com/paper/2604.12196/) (2026)
- [Inference Time Optimization with Confidence Dynamics](https://mlsanity.com/paper/2605.25244/) (2026)
- [Even GPT-5.2 Can't Count to Five: The Case for Zero-Error Horizons in Trustworthy LLMs](https://mlsanity.com/paper/2601.15714/) (2026)
- [Semantic Voting: Execution-Grounded Consensus for LLM Code Generation](https://mlsanity.com/paper/2605.08680/) (2026)
- [When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inference](https://mlsanity.com/paper/2606.08098/) (2026)
- [Fork-Think with Confidence](https://mlsanity.com/paper/2606.31484/) (2026)
- [Two-Stage Voting for Robust and Efficient Suicide Risk Detection on Social Media](https://mlsanity.com/paper/2510.08365/) (2025)
- [CASHEW: Stabilizing Multimodal Reasoning via Iterative Trajectory Aggregation](https://mlsanity.com/paper/2601.08010/) (2026)
- [R-C2: Cycle-Consistent Reinforcement Learning Improves Multimodal Reasoning](https://mlsanity.com/paper/2603.25720/) (2026)
- [Discovering Novel LLM Experts via Task-Capability Coevolution](https://mlsanity.com/paper/2604.14969/) (2026)
- [When Answers Stray from Questions: Hallucination Detection via Question-Answer Orthogonal Decomposition](https://mlsanity.com/paper/2605.14449/) (2026)
- [Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models](https://mlsanity.com/paper/2609.35875/) (2026)
- [Compare to Correct: Pairwise-Guided Logit Adjustment for LLM Test-Time Scaling](https://mlsanity.com/paper/2610.18583/) (2026)
- [Frustratingly Easy Majority Voting Using Length Conditioning](https://mlsanity.com/paper/2610.19890/) (2026)

## Abstract (arXiv)

Majority voting is considered an effective method to enhance chain-of-thought reasoning, as it selects the answer with the highest "self-consistency" among different reasoning paths (Wang et al., 2023). However, previous chain-of-thought reasoning methods typically generate only a single answer in each trial, thereby ignoring the possibility of other potential answers. As a result, these alternative answers are often overlooked in subsequent voting processes. In this work, we propose to generate ranked answers in each reasoning process and conduct ranked voting among multiple ranked answers from different responses, thereby making the overall self-consistency more reliable. Specifically, we use three ranked voting methods: Instant-runoff voting, Borda count voting, and mean reciprocal rank voting. We validate our methods on six datasets, including three multiple-choice and three open-ended question-answering tasks, using both advanced open-source and closed-source large language models. Extensive experimental results indicate that our proposed method outperforms the baselines, showcasing the potential of leveraging the information of ranked answers and using ranked voting to improve reasoning performance. The code is available at https://github.com/szu-tera/RankedVotingSC.