Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search
Not on the map. Why?
Abstract (arXiv)
Adaptive model search is path dependent: once a challenger is adopted, it becomes the reference from which later candidates are generated. A statistically unsupported replacement can therefore alter hypotheses that have not yet been proposed. We introduce Evidence-Gated Research (EGR), a statistical adoption layer for moving-incumbent search. EGR freezes each challenger before decision evidence is revealed, builds anytime-valid evidence across a predeclared set of environments, routes evidence predictably toward unresolved components, composes a persistent candidate e-value, and passes that e-value to an online controller. Under explicit conditional-validity and predictability conditions, the resulting procedure controls false discovery rate for the declared all-environment adoption target even though earlier adoptions change later challengers. In a 5,000-trajectory closed-loop benchmark, development-only e-LOND attains persistent FDR 0.621, whereas no persistent false-adoption path is observed for the audited EGR variants in that finite run. In matched replay over 600 challenger--incumbent pairs, Stagewise EGR preserves fixed-anytime alternative crossing decisions while using 56.1% less decision evidence at the representative threshold. A three-environment public-data study and a 40,000-sample controlled neural benchmark reproduce the evidence-efficiency pattern. These results identify model replacement as a distinct statistical control point in adaptive model development.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 18 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
13 of 18 references.
References · 13
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
- 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
- 2020Estimating means of bounded random variables by bettingcited by 164
- 2026Towards End-to-End Automation of AI Researchcited by 150
- 2022Game-theoretic statistics and safe anytime-valid inferencecited by 145
- 2024Hypothesis testing with e-valuescited by 70
- 2025AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-benchcited by 65
- 2025An AI system to help scientists write expert-level empirical softwarecited by 52
- 2024Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selectioncited by 13
- 2023Online multiple testing with e-valuescited by 12
- 2024Model Assessment and Selection under Temporal Distribution Shiftcited by 10
- 2023Deep anytime-valid hypothesis testingcited by 7
Built on by
Not cited yet.