When Should an AI Scientist Stop? Verifiable Experiment Steering and Refusal for Autonomous Discovery
Not on the map. Why?
Abstract (arXiv)
We present CARTOGRAPH, a verification layer for AI scientists that couples unresolved-subspace experiment steering (select), explicit ambiguity closure (resolve), and residual-based library inadequacy detection (refuse). Under a local linear-Gaussian bridge, raw unresolved projection is the isotropic unresolved Fisher-information trace, while CARTOGRAPH-A is the exact unresolved A-optimal rule; closed-form EIG and Box-Hill arise as local comparators rather than global equivalents. Across five testbeds, CARTOGRAPH-A beats raw projection 129W/0T/15L at d = 8 (p < 10^-21) in a replicated structured cascade. More distinctively, the framework tentatively identifies three out-of-library pharmacokinetic mechanisms and then revokes those identifications as residuals expose structural misfit, while one perturbed in-library control stays identified throughout. In low-dimensional pharmacokinetic and filtered EPA settings, near-ties against disagreement are predicted by theory and observed. Finally, in a retrospective audit of 40 positive claims from the published A-Lab autonomous materials system, the refuse guard flags all 4 claims later marked inconclusive under manual reanalysis while passing 32/36 confirmed claims. Code is available at https://github.com/ai4science-boed/cartograph.git
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 43 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
14 of 43 references.
References · 14
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2016Concrete Problems in AI Safetycited by 1,582
- 2018Model Cards for Model Reportingcited by 1,068
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
- 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 335
- 2023Modern Bayesian Experimental Designcited by 121
- 2019Variational Bayesian Optimal Experimental Designcited by 116
- 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 103
Built on by
Citing papers per year: 2026: 1.
1 paper.