mlsanityData to 7 Oct 2026

Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs

arXiv 2409.01281 ↗First posted 25 Aug 2024cs.CL cs.AIcited by 17

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, current methods are computationally expensive and time-consuming due to the need for numerous samplings. To address this, this paper introduces path-consistency, which leverages the confidence of earlier-generated answers to identify the most promising prefix and guide the generation of subsequent branches. By dynamically guiding the generation of subsequent branches based on this prefix, path-consistency mitigates both the errors and redundancies from random or less useful sampling in self-consistency. This approach reduces errors and redundancies from random sampling, significantly accelerating inference by minimizing token consumption. Our extensive empirical results demonstrate that path-consistency improves inference latency by up to 40.5%, while maintaining task accuracy across various tasks, including mathematical reasoning, commonsense reasoning, and symbolic reasoning.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 29 of its 33 references that are in the map. First posted on arXiv 25 Aug 2024.

Builds on

29 of 33 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  2. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
  3. 2023Self-Evaluation Guided Beam Search for Reasoning1 May
  4. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  5. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  10. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  12. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
12 more
  1. 2016Solving General Arithmetic Word Problemscited by 353
  2. 2022Large Language Models Are Reasoning Teacherscited by 318
  3. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  4. 2022Teaching Small Language Models to Reasoncited by 279
  5. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  6. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Stepcited by 144
  7. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  8. 2017Long Text Generation via Adversarial Training with Leaked Informationcited by 100
  9. 2022Coder Reviewer Reranking for Code Generationcited by 83
  10. 2019NumNet: Machine Reading Comprehension with Numerical Reasoningcited by 59
  11. 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51
  12. 2021Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoningcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 7.

17 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025Efficient Reasoning Models: A Surveycited by 93
  3. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  4. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  5. 2025Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistencycited by 12
  6. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregationcited by 11
  7. 2025C3PO: Optimized Large Language Model Cascades with Probabilistic Cost Constraints for Reasoningcited by 8
  8. 2025Self-Improvement of Language Models by Post-Training on Multi-Agent Debatecited by 7
  9. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  10. 2025Measuring Reasoning Utility in LLMs via Conditional Entropy Reductioncited by 6
  11. 2026The Path of Least Resistance: Guiding LLM Reasoning Trajectories with Prefix Consensuscited by 3
  12. 2026CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Computecited by 3
5 more
  1. 2026Reliable Chain-of-Thought via Prefix Consistencycited by 3
  2. 2025FEval-TTC: Fair Evaluation Protocol for Test-Time Computecited by 1
  3. 2026Can Decision Trees Teach Large Language Models? Distilling Verbalized Knowledge for Molecular Property Predictioncited by 1
  4. 2026HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reductioncited by 0
  5. 2026Convex Optimization for Alignment and Preference Learning on a Single GPUcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  2. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
  3. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling
  4. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models
  5. 2024Lachesis: Predicting LLM Inference Accuracy using Structural Properties of Reasoning Paths
  6. 2025Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning
  7. 2025ConSol: Sequential Probability Ratio Testing to Find Consistent LLM Reasoning Paths Efficiently
  8. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting