mlsanityData to 14 Oct 2026

When Is Availability-Aware Training Worth It? A Benchmark and Empirical Study of Interruption-Resilient Optimization Under Predictable Compute Schedules

arXiv 2609.22087 ↗First posted 22 Jun 2026eess.SP cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Training under non-stationary but predictable compute availability (satellites under eclipse, duty-cycled edge devices, power-capped datacenters) is often framed as needing specialized, availability-aware optimizers. We test that premise. We release OrbitTrace, a benchmark of 50 physics-grounded availability traces from SGP4 propagation of live two-line element sets across three orbital regimes, and ask a falsifiable question: when an availability gap interrupts training, is a specialized resumption strategy worth it, or is competent checkpoint-and-resume enough? Our central finding: when optimizer state can be preserved across a gap, the gap is essentially free. A strong checkpoint baseline that restores full optimizer state and indexes its learning-rate schedule in effective (active) time matches uninterrupted training to within data-ordering noise on CIFAR-10/ResNet-18, and exactly on a GPT-2/AdamW task. Advantages previously reported for availability-aware methods, including our own three-pillar method AAT, arise almost entirely from comparison against a weak baseline that indexes its schedule on wall-clock time. Against the strong baseline, reactive adaptation provides no advantage across stationary, optimizer-state-loss, and distribution-drift regimes. We isolate one narrow regime where it helps: for large models whose optimizer state cannot be persisted across gaps and that are interrupted by frequent, short pauses, reconstructing a decayed optimizer moment recovers only ~21% of the state-loss penalty on average (and not robustly across seeds); this vanishes for eclipse-scale gaps, where the decayed moment is indistinguishable from zero. Our contributions are a benchmark, a strong reproducible baseline protocol, and a clear characterization of when interruption-resilient optimization is worth its complexity, and when it is not.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

18 of 27 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2016Pointer Sentinel Mixture Modelscited by 3,096
  4. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  5. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  7. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  8. 2020A Metric Learning Reality Checkcited by 321
  9. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  10. 2023DiLoCo: Distributed Low-Communication Training of Language Modelscited by 121
  11. 2015Asynchronous Parallel Stochastic Gradient for Nonconvex Optimizationcited by 116
  12. 2016Asynchronous Stochastic Gradient Descent with Delay Compensationcited by 99
6 more
  1. 2022Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNscited by 49
  2. 2021Varuna: Scalable, Low-cost Training of Massive Deep Learning Modelscited by 40
  3. 2023Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templatescited by 40
  4. 2022FedSpace: An Efficient Federated Learning Framework at Satellites and Ground Stationscited by 28
  5. 2021On-Board Federated Learning for Dense LEO Constellationscited by 22
  6. 2024Energy-Aware Federated Learning in Satellite Constellationscited by 3

Built on by

Not cited yet.