mlsanityData to 7 Oct 2026

Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration

arXiv 2601.15296 ↗First posted 2 Jan 2026cs.CL cs.AIcited by 3

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that exploits entropy as a signal for branching decisions--expanding the search tree only at positions where the model exhibits genuine uncertainty. Entropy-Tree shows superior accuracy and calibration in reasoning tasks: it achieves better pass@k than Multi-chain across multiple models and datasets, and its predictive entropy demonstrates better AUROC compared to several traditional metrics. Entropy-Tree unifies efficient structured exploration and reliable uncertainty estimation within a single decoding procedure.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 34 references that are in the map. First posted on arXiv 2 Jan 2026.

Builds on

27 of 34 references.

References · 27

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  9. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  10. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  12. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
15 more
  1. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  2. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  3. 2015Weight Uncertainty in Neural Networkscited by 554
  4. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  5. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  6. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 295
  7. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  8. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 210
  9. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  10. 2024Language Models with Conformal Factuality Guaranteescited by 128
  11. 2024Large language model validity via enhanced conformal prediction methodscited by 91
  12. 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Searchcited by 75
  13. 2024DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Modelscited by 63
  14. 2025TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modelingcited by 60
  15. 2024Detecting Hallucinations in Large Language Model Generation: A Token Probability Approachcited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Where Rollouts Begin: Low-Load, High-Leverage First-Token Diversification for RLVRcited by 2
  2. 2026Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learningcited by 2
  3. 2026Uncertainty-Aware Simulation-Based Inference for Operations Research with Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning
  2. 2026Entropy-informed Decoding: Adaptive Information-Driven Branching
  3. 2025Entropy-Gated Branching for Efficient Test-Time Reasoning
  4. 2026Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
  5. 2025Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inference
  6. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
  7. 2025Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning
  8. 2025MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information