# Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting

arXiv: https://arxiv.org/abs/2410.07839 · HTML: https://mlsanity.com/paper/2410.07839/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 7 of its 35 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 7 of 7)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [Self-Influence Guided Data Reweighting for Language Model Pre-training](https://mlsanity.com/paper/2311.00913/) (2023)
- [Improving Retrieval-Augmented Large Language Models via Data Importance Learning](https://mlsanity.com/paper/2307.03027/) (2023)
- [Importance Weighting Can Help Large Language Models Self-Improve](https://mlsanity.com/paper/2408.09849/) (2024)

## Built on by (most cited first, 9 of 9)

- [CLUE: Non-parametric Verification from Experience via Hidden-State Clustering](https://mlsanity.com/paper/2510.01591/) (2025)
- [TDFlow: Agentic Workflows for Test Driven Development](https://mlsanity.com/paper/2510.23761/) (2025)
- [AlphaApollo: A System for Deep Agentic Reasoning](https://mlsanity.com/paper/2510.06261/) (2025)
- [VecCISC: Improving Confidence-Informed Self-Consistency with Reasoning Trace Clustering and Candidate Answer Selection](https://mlsanity.com/paper/2605.08070/) (2026)
- [Chopping Trees: Semantic Similarity Based Dynamic Pruning for Tree-of-Thought Reasoning](https://mlsanity.com/paper/2511.08595/) (2025)
- [Self-Consistency for LLM-Based Motion Trajectory Generation and Verification](https://mlsanity.com/paper/2603.29301/) (2026)
- [ARBITER: Reasoning Trajectory Basins and Majority Vote Failures in Test-Time Sampling](https://mlsanity.com/paper/2605.26172/) (2026)
- [Chronos: Learning Temporal Dynamics of Reasoning Chains for Test-Time Scaling](https://mlsanity.com/paper/2602.01208/) (2026)
- [Joint Consistency: A Unified Test-Time Aggregation Framework via Energy Minimization](https://mlsanity.com/paper/2605.06219/) (2026)

## Abstract (arXiv)

While large language models (LLMs) have rapidly improved their performance on a broad number of tasks, they still often fall short on reasoning tasks. As LLMs become more integrated in diverse real-world tasks, advancing their reasoning capabilities is crucial to their effectiveness in nuanced, complex problems. Wang et al.'s self-consistency framework reveals that sampling multiple rationales before taking a majority vote reliably improves model performance across various closed-answer reasoning tasks. Standard methods based on this framework aggregate the final decisions of these rationales but fail to utilize the semantic information detailed in the step-by-step reasoning paths. Our work introduces semantic self-consistency, enhancing this approach by incorporating and analyzing both the reasoning paths of these rationales in addition to their final decisions before taking a majority vote. These methods not only improve the reliability of reasoning paths but also cause more robust performance on complex reasoning tasks.