mlsanityData to 7 Oct 2026

Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models

arXiv 2402.03271 ↗First posted 5 Feb 2024cs.CL cs.AI cs.LGcited by 37

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

In the face of uncertainty, the ability to *seek information* is of fundamental importance. In many practical applications, such as medical diagnosis and troubleshooting, the information needed to solve the task is not initially given and has to be actively sought by asking follow-up questions (for example, a doctor asking a patient for more details about their symptoms). In this work, we introduce Uncertainty of Thoughts (UoT), an algorithm to augment large language models with the ability to actively seek information by asking effective questions. UoT combines 1) an *uncertainty-aware simulation approach* which enables the model to simulate possible future scenarios and how likely they are to occur, 2) *uncertainty-based rewards* motivated by information gain which incentivizes the model to seek information, and 3) a *reward propagation scheme* to select the optimal question to ask in a way that maximizes the expected reward. In experiments on medical diagnosis, troubleshooting, and the `20 Questions` game, UoT achieves an average performance improvement of 38.1% in the rate of successful task completion across multiple LLMs compared with direct prompting and also improves efficiency (i.e., the number of questions needed to complete the task). Our code has been released [here](https://github.com/zhiyuanhubj/UoT)

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 25 of its 36 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

25 of 36 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  10. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
11 more
  1. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  2. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  3. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  4. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  5. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  6. 2019End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosiscited by 70
  7. 2023Dynamic Planning with a LLMcited by 60
  8. 2020MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generationcited by 45
  9. 2021End-to-End Learning of Flowchart Grounded Task-Oriented Dialogscited by 19
  10. 2023KwaiAgents: Generalized Information-seeking Agent System with Large Language Modelscited by 14
  11. 2023Chatbots as Problem Solvers: Playing Twenty Questions with Role Reversalscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 12; 2026: 17.

37 papers.

Sort:

By area:

  1. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
  2. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 145
  3. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  4. 2024CoD, Towards an Interpretable Medical Agent using Chain of Diagnosiscited by 43
  5. 2025EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systemscited by 34
  6. 2024RethinkMCTS: Refining Erroneous Thoughts in Monte Carlo Tree Search for Code Generationcited by 25
  7. 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoningcited by 24
  8. 2025BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Designcited by 21
  9. 2024Estimating the Hallucination Rate of Generative AIcited by 20
  10. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  11. 2024Autonomous Agents for Collaborative Task under Information Asymmetrycited by 15
  12. 2025Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Miningcited by 13
25 more
  1. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 12
  2. 2025When Two LLMs Debate, Both Think They'll Wincited by 10
  3. 2025The Dialogue That Heals: A Comprehensive Evaluation of Doctor Agents' Inquiry Capabilitycited by 10
  4. 2024Can foundation models actively gather information in interactive environments to test hypotheses?cited by 7
  5. 2025Adaptive Elicitation of Latent Information Using Natural Languagecited by 7
  6. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  7. 2025Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like Peoplecited by 7
  8. 2025Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learningcited by 5
  9. 2026LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writerscited by 4
  10. 2026MedClarify: An information-seeking AI agent for medical diagnosis with case-specific follow-up questionscited by 3
  11. 2026The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Designcited by 2
  12. 2026Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?cited by 2
  13. 2026HyPAC: Cost-Efficient LLMs-Human Hybrid Annotation with PAC Error Guaranteescited by 1
  14. 2026Ask the World Before Acting: Environment Probing for Calibrated Agent World Modelscited by 1
  15. 2025Closing the Expression Gap in LLM Instructions via Socratic Questioningcited by 0
  16. 2026Adaptive Querying with AI Persona Priorscited by 0
  17. 2026BALAR : A Bayesian Agentic Loop for Active Reasoningcited by 0
  18. 2026OracleTSC: Oracle-Informed Reward Hurdle and Uncertainty Regularization for Traffic Signal Controlcited by 0
  19. 2026A Proactive Multi-Agent Dialogue Framework for Assessing Social Language Disorder Traits in Autismcited by 0
  20. 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoningcited by 0
  21. 2026CA-BED: Conversation-Aware Bayesian Experimental Designcited by 0
  22. 2026Operads for compositional reasoning in LLMscited by 0
  23. 2026Operadic consistency: a label-free signal for compositional reasoning failures in LLMscited by 0
  24. 2026Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoningcited by 0
  25. 2026Active Inference as Context Acquisition for AI Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Uncertainty-Aware Tree Search for Robust Reasoning
  2. 2025Structured Uncertainty guided Clarification for LLM Agents
  3. 2026Game of Thought: Robust Information Seeking with Large Language Models Using Game Theory
  4. 2026RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
  5. 2026Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertainty
  6. 2025Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learning
  7. 2024Plan of Thoughts: Heuristic-Guided Problem Solving with Large Language Models
  8. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study