mlsanityData to 14 Oct 2026

Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees

arXiv 2604.20500 ↗First posted 22 Apr 2026cs.LGcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting. However, in constrained domains like math and code, this strategy is compute-inefficient because it samples with replacement, repeatedly revisiting the same high-probability prefixes and duplicate completions. We propose Distinct Leaf Enumeration (DLE), a deterministic decoding method that treats truncated sampling as traversal of a pruned decoding tree and systematically enumerates distinct leaves instead of sampling with replacement. This strategy improves inference efficiency in two ways. Algorithmically, it increases coverage of the truncated search space under a fixed budget by exploring previously unvisited high-probability branches. Systemically, it reuses shared prefixes and reduces redundant token generation. Empirically, DLE explores higher-quality reasoning traces than stochastic self-consistency, yielding better performance on math, coding, and general reasoning tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 39 of its 46 references that are in the map. First posted on arXiv 22 Apr 2026.

Builds on

39 of 46 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs7 Oct
  2. 2025Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency17 Sep
  3. 2025Deep Think with Confidence21 Aug
  4. 2025Entropy-based Exploration Conduction for Multi-step Reasoning20 Mar
  5. 2025φ-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitation17 Mar
  6. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certainty25 Feb
  7. 2025Confidence Improves Self-Consistency in LLMs10 Feb
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  9. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation2 Jul
  10. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  11. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios14 Nov
  12. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
1 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  9. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  10. 2018Hierarchical Neural Story Generationcited by 1,193
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  12. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
14 more
  1. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  2. 2017Six Challenges for Neural Machine Translationcited by 335
  3. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  4. 2022A Contrastive Framework for Neural Text Generationcited by 225
  5. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  6. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 150
  7. 2022Truncation Sampling as Language Model Desmoothingcited by 123
  8. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 112
  9. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 73
  10. 2018Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translationcited by 29
  11. 2020Incremental Sampling Without Replacement for Sequence Modelscited by 23
  12. 2022Arithmetic Sampling: Parallel Diverse Decoding for Large Language Modelscited by 19
  13. 2025Efficient Beam Search for Large Language Models Using Trie-Based Decodingcited by 10
  14. 2025Decoding Memories: An Efficient Pipeline for Self-Consistency Hallucination Detectioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026V1: Unifying Generation and Self-Verification for Parallel Reasoners
  2. 2026Self-Consistency via Marginal Sharpening
  3. 2025A*-Decoding: Token-Efficient Inference Scaling
  4. 2026Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
  5. 2025The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute
  6. 2025Entropy-Gated Branching for Efficient Test-Time Reasoning
  7. 2026When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models
  8. 2026Where Draft Trees Lose Target Mass: Exit-Guided Speculative Decoding