Interpretable Probability Estimation with LLMs via Shapley Reconstruction
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Large Language Models (LLMs) demonstrate potential to estimate the probability of uncertain events, by leveraging their extensive knowledge and reasoning capabilities. This ability can be applied to support intelligent decision-making across diverse fields, such as financial forecasting and preventive healthcare. However, directly prompting LLMs for probability estimation faces significant challenges: their outputs are often noisy, and the underlying predicting process is opaque. In this paper, we propose PRISM: Probability Reconstruction via Shapley Measures, a framework that brings transparency and precision to LLM-based probability estimation. PRISM decomposes an LLM's prediction by quantifying the marginal contribution of each input factor using Shapley values. These factor-level contributions are then aggregated to reconstruct a calibrated final estimate. In our experiments, we demonstrate PRISM improves predictive accuracy over direct prompting and other baselines, across multiple domains including finance, healthcare, and agriculture. Beyond performance, PRISM provides a transparent prediction pipeline: our case studies visualize how individual factors shape the final estimate, helping build trust in LLM-based decision support systems.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 35 references that are in the map. First posted on arXiv 14 Jan 2026.
Builds on
15 of 35 references.
References · 15
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
- 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
- 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
- 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
- 2023Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Studycited by 116
- 2024TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value Estimationcited by 41
- 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Modelscited by 18
- 2025Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterizationcited by 15
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026CA-BED: Conversation-Aware Bayesian Experimental Designcited by 0
- 2026CA-BED: Conversation-Aware Bayesian Experimental Design31 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimation
- 2026From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code
- 2026IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration
- 2025Measuring What LLMs Think They Do: SHAP Faithfulness and Deployability on Financial Tabular Classification
- 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications
- 2025OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data
- 2025Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
- 2025Interpreting LLMs as Credit Risk Classifiers: Do Their Feature Explanations Align with Classical ML?