mlsanityData to 14 Oct 2026

Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule

arXiv 2606.29119 ↗First posted 28 Jun 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a pre-registered screening rule that decides, before any implementation, whether an evolutionary / population / lifecycle outer loop over neural-network parameters or structure is worth building. Such outer loops cost 10^2-10^3x their gradient inner loop, yet whether they beat a cheap single-shot alternative is usually discovered only after the expense is paid. Our rule computes, at a Phase-0 gate, a single number: the recovery R = s/G, the best single-shot gradient/curvature statistic's gain s divided by the best gain G of any cheap method evaluated, and prescribes skipping the outer loop when R >= 90%. We validate the rule on a within-lab series of pre-registered outer-loop bets (two analyzed cases plus a disclosed file drawer): in both analyzed cases a static or single-shot computation captured the effect on the project's own metric, the gate fired (R approximately 1.0 in both cases; approximately 0.95 under a stricter metric on one), and the outer loop was abandoned, including one case where a companion factorial decomposition localizes the apparent win to a static substrate change with the evolutionary lifecycle contributing no detectable gain. On one project the gate cost about 50-70 GPU-hours and screened out an estimated 400+ GPU-hours (first cell only) plus weeks of implementation, a 6-8x saving. The rule is prospectively falsifiable: a task with R < 90% where the outer loop still fails to beat single-shot would refute it.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 25 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

25 of 25 references.

References · 25

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  5. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  6. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  7. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  8. 2021Training Neural Networks with Fixed Sparse Maskscited by 214
  9. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 150
  10. 2023Compositional Exemplars for In-context Learningcited by 144
  11. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
  12. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
13 more
  1. 2018The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communitiescited by 78
  2. 2019NAS evaluation is frustratingly hardcited by 70
  3. 2020Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpotcited by 66
  4. 2025Evolution Strategies at the Hyperscalecited by 25
  5. 2020Importance of Tuning Hyperparameters of Machine Learning Algorithmscited by 20
  6. 2017On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descentcited by 16
  7. 2019Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid Methodscited by 8
  8. 2026Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Trainingcited by 8
  9. 2026Evolutionary Strategies lead to Catastrophic Forgetting in LLMscited by 7
  10. 2021Differentiable NAS Framework and Application to Ads CTR Predictioncited by 6
  11. 2026Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategiescited by 3
  12. 2021To tune or not to tune? An Approach for Recommending Important Hyperparameterscited by 2
  13. 2026Decomposing Evolutionary Mixture-of-LoRA Architectures: The Routing Lever, the Lifecycle Penalty, and a Substrate-Conditional Boundarycited by 1

Built on by

Not cited yet.