# LLM-Prior: A Framework for Knowledge-Driven Prior Elicitation and Aggregation

arXiv: https://arxiv.org/abs/2508.03766 · HTML: https://mlsanity.com/paper/2508.03766/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 13 of 13)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [LLM-SR: Scientific Equation Discovery via Programming with Large Language Models](https://mlsanity.com/paper/2404.18400/) (2024)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Learning Mixtures of Gaussians Using Diffusion Models](https://mlsanity.com/paper/2404.18869/) (2024)
- [A Federated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison](https://mlsanity.com/paper/2110.10223/) (2021)
- [Performance Prediction for Large Systems via Text-to-Text Regression](https://mlsanity.com/paper/2506.21718/) (2025)
- [Combining probability distributions: Extending the logarithmic pooling approach](https://mlsanity.com/paper/1502.04206/) (2015)
- [Mixture Density Networks for Classification with an Application to Product Bundling](https://mlsanity.com/paper/2402.05428/) (2024)

## Built on by (most cited first, 6 of 6)

- [Gaussian Joint Embeddings For Self-Supervised Representation Learning](https://mlsanity.com/paper/2603.26799/) (2026)
- [LLM-BI: Towards Fully Automated Bayesian Inference with Large Language Models](https://mlsanity.com/paper/2508.08300/) (2025)
- [Calibration, Not Compilation: Detecting and Repairing Misspecified Probabilistic Programs Written by Language Models](https://mlsanity.com/paper/2606.31630/) (2026)
- [AI4BayesCode: From Natural Language Descriptions to Validated Modular Stateful Bayesian Samplers](https://mlsanity.com/paper/2605.18476/) (2026)
- [Distribird: Literature-Informed Prior Distribution Design for Bayesian Model Calibration](https://mlsanity.com/paper/2608.11210/) (2026)
- [Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEs](https://mlsanity.com/paper/2609.32941/) (2026)

## Abstract (arXiv)

The specification of prior distributions is fundamental in Bayesian inference, yet it remains a significant bottleneck. The prior elicitation process is often a manual, subjective, and unscalable task. We propose a novel framework which leverages Large Language Models (LLMs) to automate and scale this process. We introduce textttLLMPrior, a principled operator that translates rich, unstructured contexts such as natural language descriptions, data or figures into valid, tractable probability distributions. We formalize this operator by architecturally coupling an LLM with an explicit, tractable generative model, such as a Gaussian Mixture Model (forming a LLM based Mixture Density Network), ensuring the resulting prior satisfies essential mathematical properties. We further extend this framework to multi-agent systems where Logarithmic Opinion Pooling is employed to aggregate prior distributions induced by decentralized knowledge. We present the federated prior aggregation algorithm, textttFed-LLMPrior, for aggregating distributed, context-dependent priors in a manner robust to agent heterogeneity. This work provides the foundation for a new class of tools that can potentially lower the barrier to entry for sophisticated Bayesian modeling.