Not Every Sync Is Safe: Calibrated DiLoCo Scheduling for Shared AI Infrastructure
Not on the map. Why?
Abstract (arXiv)
DiLoCo-style training reduces communication by letting learner islands train locally before occasional outer synchronization, making it attractive for fragmented industrial AI fleets where training shares hardware with latency-sensitive serving. The question for such fleets is when an outer merge is worth its system cost, and whether choosing which windows to defer matters at all. Existing scheduling studies evaluate workload-aware policies against fixed-period baselines, but most omit the control that isolates timing from budget: matched random deferral, which inherits the controller's synchronization budget but is not itself deployable. This omission is consequential: across controlled stress tests and real vLLM sidecar replays, matched random ties or beats every forecast-free policy we test, so gains reported against weaker baselines cannot be attributed to window choice. We fill this gap with Workload-Aware DiLoCo (WA-DiLoCo), a score-based controller that weighs learner progress against fleet pressure, and a calibration protocol that determines when matched random can be beaten, then demonstrate that it can. In the bursty regime where calibration exposes request-overlap structure, adding a one-step EWMA burst forecast to the online controller beats matched random in real vLLM sidecar replay, reducing SLO violations from 6.54% to 5.09% (8 of 10 seeds, p=0.021); offline Calibrated-WA, a non-deployable bound, shows the remaining headroom at 4.45% versus 6.26%. The deployable lesson remains the protocol: report real-sidecar effect-size transfer, a no-sync load match, and a matched-random envelope before claiming serving-SLO improvement.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 15 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
9 of 15 references.
References · 9
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2023DiLoCo: Distributed Low-Communication Training of Language Modelscited by 115
- 2020Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learningcited by 68
- 2024OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Trainingcited by 23
- 2025Overcoming the Communication-Performance Tradeoff in LLM Pretrainingcited by 12
- 2024FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guaranteescited by 9
- 2025NoLoCo: No-all-reduce Low Communication Training Method for Large Modelscited by 4
- 2026Decoupled DiLoCo for Resilient Distributed Pre-trainingcited by 4
- 2025MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignmentcited by 1
Built on by
Not cited yet.