# Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization

arXiv: https://arxiv.org/abs/2505.15918 · HTML: https://mlsanity.com/paper/2505.15918/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 20 of 20)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [On Verbalized Confidence Scores for LLMs](https://mlsanity.com/paper/2412.14737/) (2024)
- [From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples](https://mlsanity.com/paper/2404.07544/) (2024)
- [LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language](https://mlsanity.com/paper/2405.12856/) (2024)
- [When to Trust LLMs: Aligning Confidence with Response Quality](https://mlsanity.com/paper/2404.17287/) (2024)
- [What Are the Odds? Language Models Are Capable of Probabilistic Reasoning](https://mlsanity.com/paper/2406.12830/) (2024)
- [RuleBert: Teaching Soft Rules to Pre-trained Language Models](https://mlsanity.com/paper/2109.13006/) (2021)
- [BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models](https://mlsanity.com/paper/2404.12494/) (2024)
- [The R Package stagedtrees for Structural Learning of Stratified Staged Trees](https://mlsanity.com/paper/2004.06459/) (2020)
- [AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling](https://mlsanity.com/paper/2411.17284/) (2024)
- [bnRep: A repository of Bayesian networks from the academic literature](https://mlsanity.com/paper/2409.19158/) (2024)
- [Teaching Probabilistic Logical Reasoning to Transformers](https://mlsanity.com/paper/2305.13179/) (2023)
- [Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models](https://mlsanity.com/paper/2409.04318/) (2024)
- [Had enough of experts? Quantitative knowledge retrieval from large language models](https://mlsanity.com/paper/2402.07770/) (2024)
- [Scalability of Bayesian Network Structure Elicitation with Large Language Models: a Novel Methodology and Comparative Analysis](https://mlsanity.com/paper/2407.09311/) (2024)

## Built on by (most cited first, 15 of 15)

- [Calibrating Verbalized Confidence with Self-Generated Distractors](https://mlsanity.com/paper/2509.25532/) (2025)
- [OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data](https://mlsanity.com/paper/2510.15096/) (2025)
- [Bayesian Network Structure Discovery Using Large Language Models](https://mlsanity.com/paper/2511.00574/) (2025)
- [Interpretable Probability Estimation with LLMs via Shapley Reconstruction](https://mlsanity.com/paper/2601.09151/) (2026)
- [Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Models](https://mlsanity.com/paper/2602.10282/) (2026)
- [Humans and LLMs Diverge on Probabilistic Inferences](https://mlsanity.com/paper/2602.23546/) (2026)
- [ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models](https://mlsanity.com/paper/2605.10328/) (2026)
- [LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting](https://mlsanity.com/paper/2609.01337/) (2026)
- [Counting Clues: A Lightweight Probabilistic Baseline Can Match an LLM](https://mlsanity.com/paper/2512.12868/) (2025)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [Zero-Shot Active Feature Acquisition via LLM-Elicitation](https://mlsanity.com/paper/2606.18933/) (2026)
- [Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networks](https://mlsanity.com/paper/2607.25877/) (2026)
- [Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments](https://mlsanity.com/paper/2608.06170/) (2026)
- [PRISM-BN: A Controlled Corpus and Benchmark for Text-to-Parameterized Bayesian Network Extraction](https://mlsanity.com/paper/2609.21673/) (2026)
- [ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosis](https://mlsanity.com/paper/2610.11140/) (2026)

## Abstract (arXiv)

In this work, we evaluate the potential of Large Language Models (LLMs) in building Bayesian Networks (BNs) by approximating domain expert priors. LLMs have demonstrated potential as factual knowledge bases; however, their capability to generate probabilistic knowledge about real-world events remains understudied. We explore utilizing the probabilistic knowledge inherent in LLMs to derive probability estimates for statements regarding events and their relationships within a BN. Using LLMs in this context allows for the parameterization of BNs, enabling probabilistic modeling within specific domains. Our experiments on eighty publicly available Bayesian Networks, from healthcare to finance, demonstrate that querying LLMs about the conditional probabilities of events provides meaningful results when compared to baselines, including random and uniform distributions, as well as approaches based on next-token generation probabilities. We explore how these LLM-derived distributions can serve as expert priors to refine distributions extracted from data, especially when data is scarce. Overall, this work introduces a promising strategy for automatically constructing Bayesian Networks by combining probabilistic knowledge extracted from LLMs with real-world data. Additionally, we establish the first comprehensive baseline for assessing LLM performance in extracting probabilistic knowledge.