mlsanityData to 14 Oct 2026

UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making

arXiv 2506.17419 ↗First posted 20 Jun 2025cs.CL cs.AI cs.LGcited by 42

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

As Large Language Models (LLMs) are integrated into safety-critical applications involving sequential decision-making in the real world, it is essential to know when to trust LLM decisions. Existing LLM Uncertainty Quantification (UQ) methods are primarily designed for single-turn question-answering formats, resulting in multi-step decision-making scenarios, e.g., LLM agentic system, being underexplored. In this paper, we introduce a principled, information-theoretic framework that decomposes LLM sequential decision uncertainty into two parts: (i) internal uncertainty intrinsic to the current decision, which is focused on existing UQ methods, and (ii) extrinsic uncertainty, a Mutual-Information (MI) quantity describing how much uncertainty should be inherited from preceding decisions. We then propose UProp, an efficient and effective extrinsic uncertainty estimator that converts the direct estimation of MI to the estimation of Pointwise Mutual Information (PMI) over multiple Trajectory-Dependent Decision Processes (TDPs). UProp is evaluated over extensive multi-step decision-making benchmarks, e.g., AgentBench and HotpotQA, with state-of-the-art LLMs, e.g., GPT-4.1 and DeepSeek-V3. Experimental results demonstrate that UProp significantly outperforms existing single-turn UQ baselines equipped with thoughtful aggregation strategies. Moreover, we provide a comprehensive analysis of UProp, including sampling efficiency, potential applications, and intermediate uncertainty propagation, to demonstrate its effectiveness. Codes will be available at https://github.com/jinhaoduan/UProp.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 31 of its 41 references that are in the map. First posted on arXiv 20 Jun 2025.

Builds on

31 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SAUP: Situation Awareness Uncertainty Propagation on LLM Agent2 Dec
  2. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  3. 2023AgentBench: Evaluating LLMs as Agents7 Aug

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  9. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  10. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  11. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  12. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 323
16 more
  1. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  2. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  3. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  4. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  5. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  6. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  7. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  8. 2023Large Language Models for Robotics: A Surveycited by 133
  9. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  10. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 120
  11. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 97
  12. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  13. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentcited by 81
  14. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
  15. 2024Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generationcited by 16
  16. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 39.

42 papers.

Sort:

By area:

  1. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 20
  2. 2026Agentic Uncertainty Quantificationcited by 19
  3. 2026Agentic Confidence Calibrationcited by 15
  4. 2026TRACER: Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoningcited by 11
  5. 2026Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarkingcited by 10
  6. 2025LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systemscited by 7
  7. 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimizationcited by 7
  8. 2025Uncertainty Quantification for Retrieval-Augmented Reasoningcited by 6
  9. 2026Recurrent Confidence Chain: Temporal-Aware Uncertainty Quantification in Large Language Modelscited by 5
  10. 2025Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actionscited by 4
  11. 2026Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learningcited by 4
  12. 2026Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learningcited by 4
30 more
  1. 2026Uncertainty Propagation in LLM-Based Systemscited by 3
  2. 2026Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agentscited by 3
  3. 2026From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Modelscited by 2
  4. 2026QUIVER: A Formal Framework for Quantifying Perturbation Propagation and Bifurcation in Compound AI Systemscited by 2
  5. 2026A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Modelscited by 2
  6. 2026Bayesian Uncertainty Propagation for Agentic RAG Pipelines: A Proof-of-Concept Study on Multi-Hop Question Answeringcited by 2
  7. 2026Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agentscited by 2
  8. 2026IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generationcited by 1
  9. 2026Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carriercited by 1
  10. 2026Measuring Harness-Induced Belief Divergence in Multi-Step LLM Agentscited by 1
  11. 2026Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agentscited by 1
  12. 2026From Sequence to Structure: Relational Uncertainty Propagation for LLM Agentscited by 1
  13. 2026Monitoring Web Agents Without Internal Signals: Observable Trajectories and Key-Step Supervisioncited by 1
  14. 2026How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Methodcited by 1
  15. 2026Directional Concentration Uncertainty: A representational approach to uncertainty quantification for generative modelscited by 0
  16. 2026Proper Scoring Rules for Agentic Uncertainty Quantificationcited by 0
  17. 2026Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Frameworkcited by 0
  18. 2026Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networkscited by 0
  19. 2026Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systemscited by 0
  20. 2026PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systemscited by 0
  21. 2026CUSP: Decomposable Collective Uncertainty for Multi-Agent Multimodal Reasoningcited by 0
  22. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0
  23. 2026Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representationscited by 0
  24. 2026GRUET: Quantifying Uncertainty of Agentic Reasoning-and-Acting Processescited by 0
  25. 2026"very likely" Means "uncertain"? How LLMs Diverge from Humans in Linguistic Uncertainty Quantificationcited by 0
  26. 2026Prospective Hindsight: Self-Calibrating Reinforcement Learning via Prediction-Reality Gapscited by 0
  27. 2026Confidence Reasoning Graphs: Structured Confidence Estimation for LLM Agentscited by 0
  28. 2026From Uncertainty to Action: Learning to Steer LLM Agentscited by 0
  29. 2026Agentic Self-Awareness: Aligning Knowing, Acting, and Holdingcited by 0
  30. 2026Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chainscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey
  2. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents
  3. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models
  4. 2026Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs
  5. 2025CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought
  6. 2026Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty
  7. 2026The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
  8. 2026DiscoUQ: Structured Disagreement Analysis for Uncertainty Quantification in LLM Agent Ensembles