mlsanityData to 7 Oct 2026

Hypothesis Generation with Large Language Models

arXiv 2404.04326 ↗First posted 5 Apr 2024cs.AI cs.CL cs.CYcited by 68

Not on the map. Why?

Abstract (arXiv)

Effective generation of novel hypotheses is instrumental to scientific progress. So far, researchers have been the main powerhouse behind hypothesis generation by painstaking data analysis and thinking (also known as the Eureka moment). In this paper, we examine the potential of large language models (LLMs) to generate hypotheses. We focus on hypothesis generation based on data (i.e., labeled examples). To enable LLMs to handle arbitrarily long contexts, we generate initial hypotheses from a small number of examples and then update them iteratively to improve the quality of hypotheses. Inspired by multi-armed bandits, we design a reward function to inform the exploitation-exploration tradeoff in the update process. Our algorithm is able to generate hypotheses that enable much better predictive performance than few-shot prompting in classification tasks, improving accuracy by 31.7% on a synthetic dataset and by 13.9%, 3.3% and, 24.9% on three real-world datasets. We also outperform supervised learning by 12.8% and 11.2% on two challenging real-world datasets. Furthermore, we find that the generated hypotheses not only corroborate human-verified theories but also uncover new insights for the tasks.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 21 of its 58 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

21 of 58 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  8. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 174
  9. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  10. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
  11. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  12. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
9 more
  1. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  2. 2020"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humanscited by 86
  3. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 61
  4. 2023Goal-Driven Explainable Clustering via Language Descriptionscited by 58
  5. 2023AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Predictioncited by 46
  6. 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
  7. 2022Language Models as Inductive Reasonerscited by 32
  8. 2014The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twittercited by 21
  9. 2019Causal Effects of Brevity on Style and Success in Social Mediacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 40; 2026: 25.

68 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2025Correlated Errors in Large Language Modelscited by 42
  3. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  4. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  5. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 34
  6. 2025Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agentscited by 31
  7. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 25
  8. 2025Sparse Autoencoders for Hypothesis Generationcited by 24
  9. 2025Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedbackcited by 23
  10. 2025Sparks of Science: Hypothesis Generation Using Structured Paper Datacited by 22
  11. 2024AIGS: Generating Science from AI-Powered Automated Falsificationcited by 17
  12. 2025SciHorizon: Benchmarking AI-for-Science Readiness from Scientific Data to Large Language Modelscited by 15
38 more
  1. 2025Generating Computational Cognitive Models using Large Language Modelscited by 12
  2. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  3. 2025Formalizing Learning from Language Feedback with Provable Guaranteescited by 10
  4. 202534 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discoverycited by 9
  5. 2025A Survey of AI Scientistscited by 9
  6. 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
  7. 2025AstroAgents: A Multi-Agent AI for Hypothesis Generation from Mass Spectrometry Datacited by 8
  8. 2025Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Modelscited by 7
  9. 2026LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMscited by 7
  10. 2025Iterative Hypothesis Generation for Scientific Discovery with Monte Carlo Nash Equilibrium Self-Refining Treescited by 6
  11. 2025PriM: Principle-Inspired Material Discovery through Multi-Agent Collaborationcited by 6
  12. 2025HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generationcited by 6
  13. 2025MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated Experimental Feedbackcited by 6
  14. 2026The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Researchcited by 6
  15. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
  16. 2025WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generationcited by 5
  17. 2025SciML Agents: Write the Solver, Not the Solutioncited by 5
  18. 2025POPI: Personalizing LLMs via Optimized Natural Language Preference Inferencecited by 5
  19. 2025"The Diagram is like Guardrails": Structuring GenAI-assisted Hypotheses Exploration with an Interactive Shared Representationcited by 3
  20. 2025Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understandingcited by 3
  21. 2025AI for Scientific Discovery is a Social Problemcited by 3
  22. 2026SAGE: Agentic Framework for Interpretable and Clinically Translatable Computational Pathology Biomarker Discoverycited by 3
  23. 2026Accelerating Social Science Research via Agentic Hypothesization and Experimentationcited by 3
  24. 2026Automatically Finding Reward Model Biasescited by 3
  25. 2025Computational Basis of LLM's Decision Making in Social Simulationcited by 2
  26. 2025Large Language Models for Software Testing: A Research Roadmapcited by 2
  27. 2026CASTER: Breaking the Cost-Performance Barrier in Multi-Agent Orchestration via Context-Aware Strategy for Task Efficient Routingcited by 2
  28. 2026Autonomous Scientific Discovery via Iterative Meta-Reflectioncited by 2
  29. 2024MapExplorer: New Content Generation from Low-Dimensional Visualizationscited by 1
  30. 2025A Review on Large Language Models for Visual Analyticscited by 1
  31. 2025HypoEval: Hypothesis-Guided Evaluation for Natural Language Generationcited by 1
  32. 2025Augmenting Research Ideation with Data: An Empirical Investigation in Social Sciencecited by 1
  33. 2025Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Priorcited by 1
  34. 2025HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discoverycited by 1
  35. 2025Interestingness First Classifierscited by 1
  36. 2025Accepted with Minor Revisions: Value of AI-Assisted Scientific Writingcited by 1
  37. 2026Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Mathcited by 1
  38. 2026ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-makingcited by 1

50 most cited of 68 All 68 (JSON)