mlsanityData to 14 Oct 2026

Eliciting the Priors of Large Language Models using Iterated In-Context Learning

arXiv 2406.01860 ↗First posted 4 Jun 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

As Large Language Models (LLMs) are increasingly deployed in real-world settings, understanding the knowledge they implicitly use when making decisions is critical. One way to capture this knowledge is in the form of Bayesian prior distributions. We develop a prompt-based workflow for eliciting prior distributions from LLMs. Our approach is based on iterated learning, a Markov chain Monte Carlo method in which successive inferences are chained in a way that supports sampling from the prior distribution. We validated our method in settings where iterated learning has previously been used to estimate the priors of human participants -- causal learning, proportion estimation, and predicting everyday quantities. We found that priors elicited from GPT-4 qualitatively align with human priors in these settings. We then used the same method to elicit priors from GPT-4 for a variety of speculative events, such as the timing of the development of superhuman AI.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 20 of its 43 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

20 of 43 references.

References · 20

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023BloombergGPT: A Large Language Model for Financecited by 939
  4. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  5. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  6. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  7. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  8. 2018Task-Free Continual Learningcited by 221
  9. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 119
  10. 2020Countering Language Drift with Seeded Iterated Learningcited by 57
  11. 2023Managing extreme AI risks amid rapid progresscited by 48
  12. 2024Thousands of AI Authors on the Future of AIcited by 48
8 more
  1. 2024Automated Statistical Model Discovery with Language Modelscited by 41
  2. 2021Iterated learning for emergent systematicity in VQAcited by 21
  3. 2021Multi-label Iterated Learning for Image Classification with Label Ambiguitycited by 19
  4. 2023Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddingscited by 15
  5. 2023Imitation versus Innovation: What children can do that large language and language-and-vision models cannot (yet)?cited by 13
  6. 2020Supervised Seeded Iterated Learning for Interactive Language Learningcited by 7
  7. 2023Deep de Finetti: Recovering Topic Distributions from Large Language Modelscited by 7
  8. 2024Recovering Mental Representations from Large Language Models with Markov Chain Monte Carlocited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 12; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settingscited by 17
  2. 2025Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systemscited by 14
  3. 2025Levels of Analysis for Large Language Modelscited by 10
  4. 2026When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMscited by 10
  5. 2024Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slowcited by 8
  6. 2024Reflection-Bench: Evaluating Epistemic Agency in Large Language Modelscited by 7
  7. 2025Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representationscited by 6
  8. 2025Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Flycited by 6
  9. 2025The Other Mind: How Language Models Exhibit Human Temporal Cognitioncited by 5
  10. 2025Meta-learning ecological priors from large language models explains human learning and decision makingcited by 4
  11. 2025Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Modelscited by 2
  12. 2025Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidancecited by 1
7 more
  1. 2025Leveraging Large Language Models in Visual Speech Recognition: Model Scaling, Context-Aware Decoding, and Iterative Polishingcited by 1
  2. 2025LLM-BI: Towards Fully Automated Bayesian Inference with Large Language Modelscited by 1
  3. 2025Evolution and compression in LLMs: On the emergence of human-aligned categorizationcited by 1
  4. 2026Probing Perceptual Priors of MLLMs via Gibbs Sampling with Interpretable Generative Controlscited by 1
  5. 2025Guided Data Generation for Understanding Model Behaviorcited by 0
  6. 2026Eliciting Trustworthiness Priors of Large Language Models via Economic Gamescited by 0
  7. 2026PriorProbe: Recovering Individual-Level Priors for Personalizing Neural Networks in Facial Expression Recognitioncited by 0