mlsanityData to 14 Oct 2026

BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models

arXiv 2404.12494 ↗First posted 18 Apr 2024cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Predictive models often need to work with incomplete information in real-world tasks. Consequently, they must provide reliable probability or confidence estimation, especially in large-scale decision-making and planning tasks. Current large language models (LLMs) are insufficient for accurate estimations, but they can generate relevant factors that may affect the probabilities, produce coarse-grained probabilities when the information is more complete, and help determine which factors are relevant to specific downstream contexts. In this paper, we make use of these capabilities of LLMs to provide a significantly more accurate probabilistic estimation. We propose BIRD, a novel probabilistic inference framework that aligns a Bayesian network with LLM abductions and then estimates more accurate probabilities in a deduction step. We show BIRD provides reliable probability estimations that are 30% better than those provided directly by LLM baselines. These estimates further contribute to better and more trustworthy decision making.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

31 of 46 references.

References · 31

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  8. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  9. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  10. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  11. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  12. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
19 more
  1. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 168
  2. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  3. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 119
  4. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  5. 2023Amortizing intractable inference in large language modelscited by 97
  6. 2018A Multi-Axis Annotation Scheme for Event Temporal Relationscited by 75
  7. 2020Temporal Reasoning on Implicit Events from Distant Supervisioncited by 58
  8. 2023Training Chain-of-Thought via Latent-Variable Inferencecited by 47
  9. 2021COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentencescited by 40
  10. 2020Temporal Common Sense Acquisition with Minimal Supervisioncited by 39
  11. 2022Entailer: Answering Questions with Faithful and Truthful Chains of Reasoningcited by 38
  12. 2019Uncertain Natural Language Inferencecited by 35
  13. 2023Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?cited by 26
  14. 2022ThinkSum: Probabilistic reasoning over sets using large language modelscited by 23
  15. 2024DeLLMa: Decision Making Under Uncertainty with Large Language Modelscited by 21
  16. 2024Conceptual and Unbiased Reasoning in Language Modelscited by 19
  17. 2022Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Textscited by 18
  18. 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
  19. 2023Language Models with Rationalitycited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 7.

18 papers.

Sort:

By area:

  1. 2025Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterizationcited by 15
  2. 2025Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Modelscited by 14
  3. 2025Calibrating Verbalized Confidence with Self-Generated Distractorscited by 14
  4. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  5. 2025Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based Systemscited by 8
  6. 2024Let's Think Var-by-Var: Large Language Models Enable Ad Hoc Probabilistic Reasoningcited by 5
  7. 2025OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Datacited by 5
  8. 2025Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoningcited by 5
  9. 2024DeFine: Decision-Making with Analogical Reasoning over Factor Profilescited by 4
  10. 2026AdaptFuse: Training-Free Sequential Preference Inference via Externalized Bayesian Computationcited by 2
  11. 2025Bonsai: Interpretable Tree-Adaptive Grounded Reasoningcited by 1
  12. 2026Interpretable Probability Estimation with LLMs via Shapley Reconstructioncited by 1
6 more
  1. 2026Probabilistic Programs of Thoughtcited by 1
  2. 2025CORE: Concept-Oriented Reinforcement for Bridging the Definition-Application Gap in Mathematical Reasoningcited by 0
  3. 2026Tutorial on Reasoning for IR & IR for Reasoningcited by 0
  4. 2026Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Taskscited by 0
  5. 2026Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allowcited by 0
  6. 2026ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosiscited by 0