# Closed-loop Auto Research for Molecular Property Prediction: Discovering and Certifying Generalizable Improvements

arXiv: https://arxiv.org/abs/2606.22731 · HTML: https://mlsanity.com/paper/2606.22731/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 22 Jun 2026.

## Builds on (most cited first, 19 of 19)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search](https://mlsanity.com/paper/2504.08066/) (2025)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [FLAML: A Fast and Lightweight AutoML Library](https://mlsanity.com/paper/1911.04706/) (2019)
- [Preserving Statistical Validity in Adaptive Data Analysis](https://mlsanity.com/paper/1411.2664/) (2014)
- [DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration](https://mlsanity.com/paper/2411.15692/) (2024)
- [ResearchGym: Evaluating Language Model Agents on Real-World AI Research](https://mlsanity.com/paper/2602.15112/) (2026)
- [ADMET property prediction through combinations of molecular fingerprints](https://mlsanity.com/paper/2310.00174/) (2023)
- [Lo-Hi: Practical ML Drug Discovery Benchmark](https://mlsanity.com/paper/2310.06399/) (2023)
- [Auto Research with Specialist Agents Develops Effective and Non-Trivial Training Recipes](https://mlsanity.com/paper/2605.05724/) (2026)
- [Auto-ADMET: An Effective and Interpretable AutoML Method for Chemical ADMET Property Prediction](https://mlsanity.com/paper/2502.16378/) (2025)

## Built on by (most cited first, 6 of 6)

- [Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer](https://mlsanity.com/paper/2607.17100/) (2026)
- [The Implementation Lottery: Auditing Idea Reliability in Automated Research](https://mlsanity.com/paper/2607.26587/) (2026)
- [Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents](https://mlsanity.com/paper/2609.09219/) (2026)
- [WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents](https://mlsanity.com/paper/2609.27490/) (2026)
- [AlphaDiverse: Post-Training Local Quantitative Research Agents for Diverse Exploration in Alpha Factor Mining](https://mlsanity.com/paper/2609.29014/) (2026)
- [Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts](https://mlsanity.com/paper/2610.00314/) (2026)

## Abstract (arXiv)

Closed-loop Auto Research extends automated machine learning from fixed-dataset fitting to changing the research workflow, with language-model agents editing representations and model code and acquiring external evidence. Molecular property prediction spans many small endpoints. We ask whether this action space yields improvements generalizing beyond the validation signal selecting them. We isolate three Auto Research axes, features, models, and external evidence, under a file-level ablation lock attributing each gain to one axis over a strong baseline. Across 36 endpoints in three benchmark suites we score each selected configuration once on a held-out test whose labels the search never read. A routed pipeline taking each endpoint's best validation axis reaches positive held-out gains of 0.013, 0.011, and 0.042, the transferable axis differing by suite, data on TDC, model on Polaris, feature and model on MoleculeNet. The largest model-search gain falls from 0.041 on validation to 0.003 on test, while curated data reaches 0.022 but negative 0.019 on test, two non-transfer signatures. Curated external data raises held-out CYP2C9-substrate performance by 0.17 and half-life by 0.08, admitted through a contamination filter rejecting same-source files overlapping 64 to 89 percent of test structures, necessary but not sufficient for transfer. A matched-trial automated machine learning control did not reproduce the agent's code-level model intervention, reaching 0.006 against 0.042, and the pipeline stays competitive with an 84M-parameter pretrained 3D model on the shared training split. The experiments stay within molecular property prediction, but separating discovery from held-out certification is a domain-agnostic lesson for any closed-loop system optimising a proxy for a held-out quantity.