mlsanityData to 7 Oct 2026

Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search

arXiv 2610.01751 ↗First posted 1 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Adaptive model search is path dependent: once a challenger is adopted, it becomes the reference from which later candidates are generated. A statistically unsupported replacement can therefore alter hypotheses that have not yet been proposed. We introduce Evidence-Gated Research (EGR), a statistical adoption layer for moving-incumbent search. EGR freezes each challenger before decision evidence is revealed, builds anytime-valid evidence across a predeclared set of environments, routes evidence predictably toward unresolved components, composes a persistent candidate e-value, and passes that e-value to an online controller. Under explicit conditional-validity and predictability conditions, the resulting procedure controls false discovery rate for the declared all-environment adoption target even though earlier adoptions change later challengers. In a 5,000-trajectory closed-loop benchmark, development-only e-LOND attains persistent FDR 0.621, whereas no persistent false-adoption path is observed for the audited EGR variants in that finite run. In matched replay over 600 challenger--incumbent pairs, Stagewise EGR preserves fixed-anytime alternative crossing decisions while using 56.1% less decision evidence at the representative threshold. A three-environment public-data study and a 40,000-sample controlled neural benchmark reproduce the evidence-efficiency pattern. These results identify model replacement as a distinct statistical control point in adaptive model development.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 18 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

13 of 18 references.

References · 13

  1. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  2. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  3. 2020Estimating means of bounded random variables by bettingcited by 164
  4. 2026Towards End-to-End Automation of AI Researchcited by 150
  5. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 145
  6. 2024Hypothesis testing with e-valuescited by 70
  7. 2025AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-benchcited by 65
  8. 2025An AI system to help scientists write expert-level empirical softwarecited by 52
  9. 2024Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selectioncited by 13
  10. 2023Online multiple testing with e-valuescited by 12
  11. 2024Model Assessment and Selection under Temporal Distribution Shiftcited by 10
  12. 2023Deep anytime-valid hypothesis testingcited by 7
1 more
  1. 2026Improving online FDR procedures via online analogs of e-closure and compound e-valuescited by 2

Built on by

Not cited yet.