mlsanityData to 7 Oct 2026

Large Language Models for Automated Open-domain Scientific Hypotheses Discovery

arXiv 2309.02726 ↗First posted 6 Sep 2023cs.CL cs.AIcited by 100

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Hypothetical induction is recognized as the main reasoning type when scientists make observations about the world and try to propose hypotheses to explain those observations. Past research on hypothetical induction is under a constrained setting: (1) the observation annotations in the dataset are carefully manually handpicked sentences (resulting in a close-domain setting); and (2) the ground truth hypotheses are mostly commonsense knowledge, making the task less challenging. In this work, we tackle these problems by proposing the first dataset for social science academic hypotheses discovery, with the final goal to create systems that automatically generate valid, novel, and helpful scientific hypotheses, given only a pile of raw web corpus. Unlike previous settings, the new dataset requires (1) using open-domain data (raw web corpus) as observations; and (2) proposing hypotheses even new to humanity. A multi-module framework is developed for the task, including three different feedback mechanisms to boost performance, which exhibits superior performance in terms of both GPT-4 based and expert-based evaluation. To the best of our knowledge, this is the first work showing that LLMs are able to generate novel (''not existing in literature'') and valid (''reflecting reality'') scientific hypotheses.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Builds on 15 of its 38 references that are in the map. First posted on arXiv 6 Sep 2023.

Builds on

15 of 38 references.

References · 15

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  7. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  8. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  9. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  10. 2023Emergent autonomous scientific research capabilities of large language modelscited by 127
  11. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  12. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 61
3 more
  1. 2022Language Models as Inductive Reasonerscited by 32
  2. 2023Logical Reasoning over Natural Language as Knowledge Representation: A Surveycited by 32
  3. 2020Improving Event Duration Prediction via Time-aware Pre-trainingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 21; 2025: 51; 2026: 27.

100 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
  3. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  4. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  5. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  6. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  7. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  8. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 94
  9. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
  10. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 74
  11. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  12. 2024Hypothesis Generation with Large Language Modelscited by 68
38 more
  1. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 68
  2. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  3. 2024MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agentscited by 64
  4. 2025ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decompositioncited by 57
  5. 2024Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideascited by 52
  6. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimizationcited by 52
  7. 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
  8. 2025ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubricscited by 44
  9. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  10. 2024IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedbackcited by 40
  11. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  12. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  13. 2024Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluationcited by 33
  14. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Researchcited by 29
  15. 2025Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedbackcited by 23
  16. 2024Planning and Editing What You Retrieve for Enhanced Tool Learningcited by 21
  17. 2024Explaining Datasets in Words: Statistical Models with Natural Language Parameterscited by 21
  18. 2024Literature Meets Data: A Synergistic Approach to Hypothesis Generationcited by 19
  19. 2024IdeaBench: Benchmarking Large Language Models for Research Idea Generationcited by 19
  20. 2025All That Glitters is Not Novel: Plagiarism in AI Generated Researchcited by 17
  21. 2025How Far Are AI Scientists from Changing the World?cited by 15
  22. 2025ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologiescited by 14
  23. 2025MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Searchcited by 13
  24. 2025AutoLabs: Cognitive Multi-Agent Systems with Self-Correction for Autonomous Chemical Experimentationcited by 13
  25. 2025Spark: A System for Scientifically Creative Idea Generationcited by 10
  26. 2025MIR: Methodology Inspiration Retrieval for Scientific Research Problemscited by 10
  27. 2025HypER: Literature-grounded Hypothesis Generation and Distillation with Provenancecited by 10
  28. 2025BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generationcited by 10
  29. 202534 Examples of LLM Applications in Materials Science and Chemistry: Towards Automation, Assistants, Agents, and Accelerated Scientific Discoverycited by 9
  30. 2025Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluatorcited by 9
  31. 2025A Survey of AI Scientistscited by 9
  32. 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
  33. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  34. 2025Automatic Evaluation Metrics for Artificially Generated Scientific Researchcited by 8
  35. 2025LLM-REVal: Can We Trust LLM Reviewers Yet?cited by 8
  36. 2026Is this Idea Novel? An Automated Benchmark for Judgment of Research Ideascited by 8
  37. 2024Creativity in AI: Progresses and Challengescited by 7
  38. 2025Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Modelscited by 7

50 most cited of 100 All 100 (JSON)