# Hypothesis Generation with Large Language Models

arXiv: https://arxiv.org/abs/2404.04326 · HTML: https://mlsanity.com/paper/2404.04326/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 58 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [SciMON: Scientific Inspiration Machines Optimized for Novelty](https://mlsanity.com/paper/2305.14259/) (2023)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Instruction Induction: From Few Examples to Natural Language Task Descriptions](https://mlsanity.com/paper/2205.10782/) (2022)
- [Large Language Models for Automated Open-domain Scientific Hypotheses Discovery](https://mlsanity.com/paper/2309.02726/) (2023)
- [Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement](https://mlsanity.com/paper/2310.08559/) (2023)
- ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020)
- [Goal Driven Discovery of Distributional Differences via Language Descriptions](https://mlsanity.com/paper/2302.14233/) (2023)
- [Goal-Driven Explainable Clustering via Language Descriptions](https://mlsanity.com/paper/2305.13749/) (2023)
- [AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction](https://mlsanity.com/paper/2305.09620/) (2023)
- [Large Language Models are Zero Shot Hypothesis Proposers](https://mlsanity.com/paper/2311.05965/) (2023)
- [Language Models as Inductive Reasoners](https://mlsanity.com/paper/2212.10923/) (2022)
- [The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter](https://mlsanity.com/paper/1405.1438/) (2014)
- [Causal Effects of Brevity on Style and Success in Social Media](https://mlsanity.com/paper/1909.02565/) (2019)

## Built on by (most cited first, 30 of 71)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Correlated Errors in Large Language Models](https://mlsanity.com/paper/2506.07962/) (2025)
- [Automated Hypothesis Validation with Agentic Sequential Falsifications](https://mlsanity.com/paper/2502.09858/) (2025)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation](https://mlsanity.com/paper/2504.11524/) (2025)
- [Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents](https://mlsanity.com/paper/2502.16069/) (2025)
- [PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaboration](https://mlsanity.com/paper/2505.15047/) (2025)
- [Sparse Autoencoders for Hypothesis Generation](https://mlsanity.com/paper/2502.04382/) (2025)
- [Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback](https://mlsanity.com/paper/2501.03916/) (2025)
- [Sparks of Science: Hypothesis Generation Using Structured Paper Data](https://mlsanity.com/paper/2504.12976/) (2025)
- [AIGS: Generating Science from AI-Powered Automated Falsification](https://mlsanity.com/paper/2411.11910/) (2024)
- [SciHorizon: Benchmarking AI-for-Science Readiness from Scientific Data to Large Language Models](https://mlsanity.com/paper/2503.13503/) (2025)
- [Generating Computational Cognitive Models using Large Language Models](https://mlsanity.com/paper/2502.00879/) (2025)
- [Formalizing Learning from Language Feedback with Provable Guarantees](https://mlsanity.com/paper/2506.10341/) (2025)
- [LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs](https://mlsanity.com/paper/2605.24043/) (2026)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [A Survey of AI Scientists](https://mlsanity.com/paper/2510.23045/) (2025)
- [34 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discovery](https://mlsanity.com/paper/2505.03049/) (2025)
- [Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models](https://mlsanity.com/paper/2505.14599/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry](https://mlsanity.com/paper/2411.15221/) (2024)
- [AstroAgents: A Multi-Agent AI for Hypothesis Generation from Mass Spectrometry Data](https://mlsanity.com/paper/2503.23170/) (2025)
- [PriM: Principle-Inspired Material Discovery through Multi-Agent Collaboration](https://mlsanity.com/paper/2504.08810/) (2025)
- [HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation](https://mlsanity.com/paper/2505.00038/) (2025)
- [Iterative Hypothesis Generation for Scientific Discovery with Monte Carlo Nash Equilibrium Self-Refining Trees](https://mlsanity.com/paper/2503.19309/) (2025)
- [MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated Experimental Feedback](https://mlsanity.com/paper/2505.17873/) (2025)
- [SciML Agents: Write the Solver, Not the Solution](https://mlsanity.com/paper/2509.09936/) (2025)
- [POPI: Personalizing LLMs via Optimized Natural Language Preference Inference](https://mlsanity.com/paper/2510.17881/) (2025)
- [The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Research](https://mlsanity.com/paper/2602.18458/) (2026)
- [WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generation](https://mlsanity.com/paper/2503.19065/) (2025)

## Abstract (arXiv)

Effective generation of novel hypotheses is instrumental to scientific progress. So far, researchers have been the main powerhouse behind hypothesis generation by painstaking data analysis and thinking (also known as the Eureka moment). In this paper, we examine the potential of large language models (LLMs) to generate hypotheses. We focus on hypothesis generation based on data (i.e., labeled examples). To enable LLMs to handle arbitrarily long contexts, we generate initial hypotheses from a small number of examples and then update them iteratively to improve the quality of hypotheses. Inspired by multi-armed bandits, we design a reward function to inform the exploitation-exploration tradeoff in the update process. Our algorithm is able to generate hypotheses that enable much better predictive performance than few-shot prompting in classification tasks, improving accuracy by 31.7% on a synthetic dataset and by 13.9%, 3.3% and, 24.9% on three real-world datasets. We also outperform supervised learning by 12.8% and 11.2% on two challenging real-world datasets. Furthermore, we find that the generated hypotheses not only corroborate human-verified theories but also uncover new insights for the tasks.