# LLM sequential decision making under uncertainty in biochemical domains

arXiv: https://arxiv.org/abs/2609.33061 · HTML: https://mlsanity.com/paper/2609.33061/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Towards a Science of Scaling Agent Systems](https://mlsanity.com/paper/2512.08296/) (2025)
- [Robin: A multi-agent system for automating scientific discovery](https://mlsanity.com/paper/2505.13400/) (2025)
- [Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective](https://mlsanity.com/paper/2406.00793/) (2024)
- [LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities](https://mlsanity.com/paper/2504.16078/) (2025)
- [Tx-LLM: A Large Language Model for Therapeutics](https://mlsanity.com/paper/2406.06316/) (2024)
- [Sparks: Multi-Agent Artificial Intelligence Model Discovers Protein Design Principles](https://mlsanity.com/paper/2504.19017/) (2025)
- [Calibrating Verbalized Probabilities for Large Language Models](https://mlsanity.com/paper/2410.06707/) (2024)
- [When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMs](https://mlsanity.com/paper/2602.06286/) (2026)
- [Behavioral Economics of AI: LLM Biases and Corrections](https://mlsanity.com/paper/2602.09362/) (2026)
- [How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks](https://mlsanity.com/paper/2608.14905/) (2026)
- [Comparing Exploration-Exploitation Strategies of LLMs and Humans: Insights from Standard Multi-armed Bandit Experiments](https://mlsanity.com/paper/2505.09901/) (2025)
- [Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity](https://mlsanity.com/paper/2604.17609/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used to drive scientific discovery. Understanding how LLMs make decisions from new data and memory of the literature is vital before trusting them to design experiments under tight experimental budgets. However, their decision strategies are invisible in the current performance scores used to evaluate research agents. Here, we benchmark five frontier LLMs in a Bayesian Optimization setting against published statistical baselines on seven combinatorial datasets spanning protein engineering, reaction optimization, molecular design, peptide self-assembly, and catalysis. Performance is paired with direct measurements of model beliefs and actions, enabling highly resolved behavior analysis. A prompt ablation that progressively strips context separates memorization from chemical reasoning and from bare categorical optimization. Prior chemical knowledge helps in expectation, but with high variance and occasionally even harms performance. No configuration tested decisively beats a mean statistical baseline across domains. Belief-movement and Martingale diagnostics, corrected here for a measurement-noise bias that mislabels rational agents as irrational, show that models overreact to incoming data rather than entrenching on their priors in the contexts studied here. Interestingly, while LLM actions are exploitative, models sincerely intend to explore and consistently act on that intent. This failure is a competence gap arising from context-stickiness. Removing in-context history restores exploration, indicating that priors and data must be decoupled to achieve effective LLM-driven discovery.