# Can AI Scientist Agents Learn from Lab-in-the-Loop Feedback? Evidence from Iterative Perturbation Discovery

arXiv: https://arxiv.org/abs/2603.26177 · HTML: https://mlsanity.com/paper/2603.26177/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 8 of 8)

- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [EvoPrompting: Language Models for Code-Level Neural Architecture Search](https://mlsanity.com/paper/2302.14838/) (2023)
- [Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective](https://mlsanity.com/paper/2406.00793/) (2024)
- [BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments](https://mlsanity.com/paper/2405.17631/) (2024)
- [Can In-context Learning Really Generalize to Out-of-distribution Tasks?](https://mlsanity.com/paper/2410.09695/) (2024)
- [LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?](https://mlsanity.com/paper/2509.21403/) (2025)
- [POLO: Preference-Guided Multi-Turn Reinforcement Learning for Lead Optimization](https://mlsanity.com/paper/2509.21737/) (2025)
- [CellCLIP -- Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning](https://mlsanity.com/paper/2506.06290/) (2025)

## Built on by (most cited first, 1 of 1)

- [Prior or Feedback? What an LLM Uses When Adapting Neural Operators](https://mlsanity.com/paper/2610.12325/) (2026)

## Abstract (arXiv)

Recent work has questioned whether large language models (LLMs) can perform genuine in-context learning (ICL) for scientific experimental design, with prior studies suggesting that LLM-based agents exhibit no sensitivity to experimental feedback. We shed new light on this question by carrying out 800 independently replicated experiments on iterative perturbation discovery in Cell Painting high-content screening. We compare an LLM agent that iteratively updates its hypotheses using experimental feedback to a zero-shot baseline that relies solely on pretraining knowledge retrieval. Access to feedback yields a +53.4% increase in discoveries per feature on average (p = 0.003). To test whether this improvement arises from genuine feedback-driven learning rather than prompt-induced recall of pretraining knowledge, we introduce a random feedback control in which hit/miss labels are permuted. Under this control, the performance gain disappears, indicating that the observed improvement depends on the structure of the feedback signal (+13.0 hits, p = 0.003). We further examine how model capability affects feedback utilization. Upgrading from Claude Sonnet 4.5 to 4.6 reduces gene hallucination rates from ~33%--45% to ~3--9%, converting a non-significant ICL effect (+0.8, p = 0.32) into a large and highly significant improvement (+11.0, p=0.003) for the best ICL strategy. These results suggest that effective in-context learning from experimental feedback emerges only once models reach a sufficient capability threshold.