# Entropy-based Exploration Conduction for Multi-step Reasoning

arXiv: https://arxiv.org/abs/2503.15848 · HTML: https://mlsanity.com/paper/2503.15848/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 36 of its 48 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Demystifying Long Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2502.03373/) (2025)
- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [Chain of Thought Empowers Transformers to Solve Inherently Serial Problems](https://mlsanity.com/paper/2402.12875/) (2024)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Chain-of-Thought Reasoning Without Prompting](https://mlsanity.com/paper/2402.10200/) (2024)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://mlsanity.com/paper/2502.09601/) (2025)
- [Unsupervised Commonsense Question Answering with Self-Talk](https://mlsanity.com/paper/2004.05483/) (2020)
- [Distilling System 2 into System 1](https://mlsanity.com/paper/2407.06023/) (2024)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)
- [On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks](https://mlsanity.com/paper/2402.08115/) (2024)
- [Chain of Thoughtlessness? An Analysis of CoT in Planning](https://mlsanity.com/paper/2405.04776/) (2024)
- [GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems](https://mlsanity.com/paper/2310.12397/) (2023)
- [ConvGQR: Generative Query Reformulation for Conversational Search](https://mlsanity.com/paper/2305.15645/) (2023)
- [Can Language Models Learn to Skip Steps?](https://mlsanity.com/paper/2411.01855/) (2024)
- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs](https://mlsanity.com/paper/2404.15676/) (2024)
- [Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation](https://mlsanity.com/paper/2401.15449/) (2024)
- [Understanding Chain-of-Thought in LLMs through Information Theory](https://mlsanity.com/paper/2411.11984/) (2024)

## Built on by (most cited first, 29 of 29)

- [SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning](https://mlsanity.com/paper/2506.01713/) (2025)
- [A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law](https://mlsanity.com/paper/2505.02665/) (2025)
- [GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts](https://mlsanity.com/paper/2601.05110/) (2026)
- [REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding](https://mlsanity.com/paper/2511.13026/) (2025)
- [UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations](https://mlsanity.com/paper/2507.07030/) (2025)
- [OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG](https://mlsanity.com/paper/2601.09028/) (2026)
- [Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation](https://mlsanity.com/paper/2602.09305/) (2026)
- [Revisiting Text Ranking in Deep Research](https://mlsanity.com/paper/2602.21456/) (2026)
- [Fast Quiet-STaR: Thinking Without Thought Tokens](https://mlsanity.com/paper/2505.17746/) (2025)
- [RAST: Reasoning Activation in LLMs via Small-model Transfer](https://mlsanity.com/paper/2506.15710/) (2025)
- [Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning](https://mlsanity.com/paper/2510.01444/) (2025)
- [ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting](https://mlsanity.com/paper/2602.19969/) (2026)
- [StaRPO: Stability-Augmented Reinforcement Policy Optimization](https://mlsanity.com/paper/2604.08905/) (2026)
- [Efficient Reasoning with Balanced Thinking](https://mlsanity.com/paper/2603.12372/) (2026)
- [Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2606.02020/) (2026)
- [The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?](https://mlsanity.com/paper/2604.06192/) (2026)
- [SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models](https://mlsanity.com/paper/2510.02648/) (2025)
- [LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning](https://mlsanity.com/paper/2601.10775/) (2026)
- [Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2601.13115/) (2026)
- [Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rank](https://mlsanity.com/paper/2604.09704/) (2026)
- [Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees](https://mlsanity.com/paper/2604.20500/) (2026)
- [CroSearch-R1: Better Leveraging Cross-lingual Knowledge for Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.25182/) (2026)
- [Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization](https://mlsanity.com/paper/2605.08978/) (2026)
- [WXImpactBench: A Disruptive Weather Impact Understanding Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2505.20249/) (2025)
- [LogitsCoder: Towards Efficient Chain-of-Thought Path Search via Logits Preference Decoding for Code Generation](https://mlsanity.com/paper/2602.14054/) (2026)
- [Avoiding Overthinking and Underthinking: Curriculum-Aware Budget Scheduling for LLMs](https://mlsanity.com/paper/2604.19780/) (2026)
- [Entropy-informed Decoding: Adaptive Information-Driven Branching](https://mlsanity.com/paper/2605.09745/) (2026)
- [BrailleBench: Investigating Multi-Criteria Braille Comprehension in Large Language Models](https://mlsanity.com/paper/2608.27268/) (2026)
- [REWIND: Uncertainty-Guided Trajectory Repair for Test-Time Reasoning](https://mlsanity.com/paper/2610.19561/) (2026)

## Abstract (arXiv)

Multi-step processes via large language models (LLMs) have proven effective for solving complex reasoning tasks. However, the depth of exploration of the reasoning procedure can significantly affect the task performance. Existing methods to automatically decide the depth often lead to high cost and a lack of flexibility. To address these issues, we propose Entropy-based Exploration Depth Conduction (Entro-duction), a novel method that dynamically adjusts the exploration depth during multi-step reasoning by monitoring LLM's output entropy and variance entropy. We employ these two features to capture the model's uncertainty of the current step and the fluctuation of uncertainty across consecutive reasoning steps. Based on the observed entropy changes, the LLM selects whether to deepen, expand, or stop exploration according to the probability, which facilitates the trade-off between the reasoning accuracy and exploration effectiveness. Experimental results across four benchmark datasets demonstrate the efficacy of Entro-duction.