mlsanityData to 7 Oct 2026

Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language

arXiv 2306.02797 ↗First posted 5 Jun 2023cs.CL cs.AI cs.LGcited by 21

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts. We introduce a model of inductive learning that seeks to be human-like in that sense. It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood. By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 30 of its 93 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

30 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022PAL: Program-aided Language Models18 Nov

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  7. 2023Teaching Large Language Models to Self-Debugcited by 963
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  9. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  10. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  11. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  12. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
17 more
  1. 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 389
  2. 2019Abductive Commonsense Reasoningcited by 300
  3. 2022Generating Sequences by Learning to Self-Correctcited by 271
  4. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  5. 2017RobustFill: Neural Program Learning under Noisy I/Ocited by 124
  6. 2021Skill Induction and Planning with Latent Languagecited by 99
  7. 2017Learning with Latent Languagecited by 75
  8. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  9. 2021Inductive logic programming at 30cited by 46
  10. 2021Leveraging Language to Learn Program Abstractions and Search Heuristicscited by 44
  11. 2021Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planningcited by 43
  12. 2022Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inferencecited by 40
  13. 2016Inference Compilation and Universal Probabilistic Programmingcited by 31
  14. 2020Learning abstract structure for drawing by efficient motor program inductioncited by 23
  15. 2022Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machinescited by 20
  16. 2019Bayesian Synthesis of Probabilistic Programs for Automatic Data Modelingcited by 19
  17. 2020Program Synthesis with Pragmatic Communicationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 8; 2026: 2.

21 papers.

Sort:

By area:

  1. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  2. 2025Bayesian Teaching Enables Probabilistic Reasoning in Large Language Modelscited by 36
  3. 2023Active Preference Inference using Language Models and Probabilistic Reasoningcited by 29
  4. 2024Is Programming by Example solved by LLMs?cited by 27
  5. 2024Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoningcited by 23
  6. 2024On Large Language Models in National Security Applicationscited by 18
  7. 2024Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitationcited by 18
  8. 2024SymbolicAI: A framework for logic-based approaches combining generative models and solverscited by 15
  9. 2024Loose LIPS Sink Ships: Asking Questions in Battleship with Language-Informed Program Samplingcited by 12
  10. 2024Does learning the right latent variables necessarily improve in-context learning?cited by 12
  11. 2025BASIL: Bayesian Assessment of Sycophancy in LLMscited by 11
  12. 2025Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like Peoplecited by 7
9 more
  1. 2025Program Synthesis via Test-Time Transductioncited by 4
  2. 2025On Language Models' Sensitivity to Suspicious Coincidencescited by 2
  3. 2023Concept-Oriented Deep Learning with Large Language Modelscited by 1
  4. 2024Parameter Choice and Neuro-Symbolic Approaches for Deep Domain-Invariant Learningcited by 1
  5. 2025Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidancecited by 1
  6. 2026Wild Guesses and Mild Guesses in Active Concept Learningcited by 1
  7. 2025Color as the Impetus: Transforming Few-Shot Learnercited by 0
  8. 2025RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Modelscited by 0
  9. 2026Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Induction and Inquiry via Probabilistic Reasoning over Language and Code
  2. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
  3. 2024On the Role of Model Prior in Real-World Inductive Reasoning
  4. 2022Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
  5. 2023Hypothesis Search: Inductive Reasoning with Language Models
  6. 2023Inductive reasoning in humans and large language models
  7. 2024Generating Diverse Hypotheses for Inductive Reasoning
  8. 2024An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models