# Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search

arXiv: https://arxiv.org/abs/2608.15669 · HTML: https://mlsanity.com/paper/2608.15669/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 96 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 30 of 39)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models](https://mlsanity.com/paper/2410.09671/) (2024)
- [The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas](https://mlsanity.com/paper/2506.20803/) (2025)
- [A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?](https://mlsanity.com/paper/2402.05015/) (2024)
- [A Tutorial on LLM Reasoning: Relevant Methods behind ChatGPT o1](https://mlsanity.com/paper/2502.10867/) (2025)
- [Understanding Emergent In-Context Learning from a Kernel Regression Perspective](https://mlsanity.com/paper/2305.12766/) (2023)
- [Bayesian Optimization of Catalysis With In-Context Learning](https://mlsanity.com/paper/2304.05341/) (2023)
- [Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs](https://mlsanity.com/paper/2509.17998/) (2025)
- [ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research](https://mlsanity.com/paper/2606.07591/) (2026)
- [AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisation](https://mlsanity.com/paper/2201.12570/) (2022)
- [LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization](https://mlsanity.com/paper/2505.14756/) (2025)
- [LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?](https://mlsanity.com/paper/2509.21403/) (2025)
- [LLM-Enhanced Bayesian Optimization for Efficient Analog Layout Constraint Generation](https://mlsanity.com/paper/2406.05250/) (2024)

## Abstract (arXiv)

Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation. Compared to LLM-only reflection or traditional statistical search across these domains, LDM achieves a 2.4× greater reduction in validation BPB, an 18.2% relative decrease in binding energy, and more than 60% relative gains in molecular multi-objective performance. These results suggests that LDM could serve as a general-purpose discovery engine for effective search over open-ended hypothesis spaces.