# Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language

arXiv: https://arxiv.org/abs/2306.02797 · HTML: https://mlsanity.com/paper/2306.02797/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 30 of its 93 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [Abductive Commonsense Reasoning](https://mlsanity.com/paper/1908.05739/) (2019)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [RobustFill: Neural Program Learning under Noisy I/O](https://mlsanity.com/paper/1703.07469/) (2017)
- [Skill Induction and Planning with Latent Language](https://mlsanity.com/paper/2110.01517/) (2021)
- [Learning with Latent Language](https://mlsanity.com/paper/1711.00482/) (2017)
- [Reasoning About Pragmatics with Neural Listeners and Speakers](https://mlsanity.com/paper/1604.00562/) (2016)
- [Inductive logic programming at 30](https://mlsanity.com/paper/2102.10556/) (2021)
- [Leveraging Language to Learn Program Abstractions and Search Heuristics](https://mlsanity.com/paper/2106.11053/) (2021)
- [Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning](https://mlsanity.com/paper/2107.12544/) (2021)
- [Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference](https://mlsanity.com/paper/2211.11875/) (2022)
- [Inference Compilation and Universal Probabilistic Programming](https://mlsanity.com/paper/1610.09900/) (2016)
- [Learning abstract structure for drawing by efficient motor program induction](https://mlsanity.com/paper/2008.03519/) (2020)
- [Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines](https://mlsanity.com/paper/2205.11558/) (2022)
- [Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling](https://mlsanity.com/paper/1907.06249/) (2019)
- [Program Synthesis with Pragmatic Communication](https://mlsanity.com/paper/2007.05060/) (2020)

## Built on by (most cited first, 22 of 22)

- [WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment](https://mlsanity.com/paper/2402.12275/) (2024)
- [Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models](https://mlsanity.com/paper/2503.17523/) (2025)
- [Active Preference Inference using Language Models and Probabilistic Reasoning](https://mlsanity.com/paper/2312.12009/) (2023)
- [Is Programming by Example solved by LLMs?](https://mlsanity.com/paper/2406.08316/) (2024)
- [Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning](https://mlsanity.com/paper/2402.06025/) (2024)
- [On Large Language Models in National Security Applications](https://mlsanity.com/paper/2407.03453/) (2024)
- [Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation](https://mlsanity.com/paper/2408.16228/) (2024)
- [SymbolicAI: A framework for logic-based approaches combining generative models and solvers](https://mlsanity.com/paper/2402.00854/) (2024)
- [Does learning the right latent variables necessarily improve in-context learning?](https://mlsanity.com/paper/2405.19162/) (2024)
- [BASIL: Bayesian Assessment of Sycophancy in LLMs](https://mlsanity.com/paper/2508.16846/) (2025)
- [Loose LIPS Sink Ships: Asking Questions in Battleship with Language-Informed Program Sampling](https://mlsanity.com/paper/2402.19471/) (2024)
- [Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People](https://mlsanity.com/paper/2510.20886/) (2025)
- [Program Synthesis via Test-Time Transduction](https://mlsanity.com/paper/2509.17393/) (2025)
- [On Language Models' Sensitivity to Suspicious Coincidences](https://mlsanity.com/paper/2504.09387/) (2025)
- [Concept-Oriented Deep Learning with Large Language Models](https://mlsanity.com/paper/2306.17089/) (2023)
- [Parameter Choice and Neuro-Symbolic Approaches for Deep Domain-Invariant Learning](https://mlsanity.com/paper/2410.06235/) (2024)
- [Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidance](https://mlsanity.com/paper/2503.15886/) (2025)
- [Wild Guesses and Mild Guesses in Active Concept Learning](https://mlsanity.com/paper/2602.06818/) (2026)
- [Color as the Impetus: Transforming Few-Shot Learner](https://mlsanity.com/paper/2507.22136/) (2025)
- [RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Models](https://mlsanity.com/paper/2510.19698/) (2025)
- [Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition](https://mlsanity.com/paper/2603.07911/) (2026)
- [Hypothesis-guided discovery of cognitive algorithms via program refinement](https://mlsanity.com/paper/2610.02523/) (2026)

## Abstract (arXiv)

A core tension in models of concept learning is that the model must carefully balance the tractability of inference against the expressivity of the hypothesis class. Humans, however, can efficiently learn a broad range of concepts. We introduce a model of inductive learning that seeks to be human-like in that sense. It implements a Bayesian reasoning process where a language model first proposes candidate hypotheses expressed in natural language, which are then re-weighed by a prior and a likelihood. By estimating the prior from human data, we can predict human judgments on learning problems involving numbers and sets, spanning concepts that are generative, discriminative, propositional, and higher-order.