# FedQueue: Queue-Aware Federated Learning for Cross-Facility HPC Training

arXiv: https://arxiv.org/abs/2605.02125 · HTML: https://mlsanity.com/paper/2605.02125/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 23 of 23)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Federated Learning with Buffered Asynchronous Aggregation](https://mlsanity.com/paper/2106.06639/) (2021)
- [LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset](https://mlsanity.com/paper/2402.09391/) (2024)
- [Stochastic Controlled Averaging for Federated Learning with Communication Compression](https://mlsanity.com/paper/2308.08165/) (2023)
- [When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions](https://mlsanity.com/paper/2306.15546/) (2023)
- [A Unified Analysis of Federated Learning with Arbitrary Client Participation](https://mlsanity.com/paper/2205.13648/) (2022)
- [FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update](https://mlsanity.com/paper/2312.05770/) (2023)
- [Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraints](https://mlsanity.com/paper/2205.06730/) (2022)
- [FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction](https://mlsanity.com/paper/2006.07273/) (2020)
- [APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2202.03672/) (2022)
- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates](https://mlsanity.com/paper/2206.10189/) (2022)
- [FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices using a Computing Power Aware Scheduler](https://mlsanity.com/paper/2309.14675/) (2023)
- [FedFa: A Fully Asynchronous Training Paradigm for Federated Learning](https://mlsanity.com/paper/2404.11015/) (2024)
- [Predicting batch queue job wait times for informed scheduling of urgent HPC workloads](https://mlsanity.com/paper/2204.13543/) (2022)
- [Globus Service Enhancements for Exascale Applications and Facilities](https://mlsanity.com/paper/2503.22981/) (2025)
- [Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset](https://mlsanity.com/paper/2508.01675/) (2025)
- [A performance analysis of VM-based Trusted Execution Environments for Confidential Federated Learning](https://mlsanity.com/paper/2501.11558/) (2025)
- [Scalable Cross-Facility Federated Learning for Scientific Foundation Models on Multiple Supercomputers](https://mlsanity.com/paper/2603.19544/) (2026)

## Abstract (arXiv)

Federated learning (FL) across multiple HPC facilities faces stochastic admission delays from batch schedulers that dominate wall-clock time. Synchronous FL suffers from severe stragglers, while asynchronous FL accumulates stale updates when queues spike. We propose FedQueue, a queue-aware FL protocol that incorporates scheduler delays directly into training and aggregation, which (i) predicts per-facility queue delays online to budget local work, (ii) applies cutoff-based admission that buffers late arrivals to bound staleness, and (iii) performs staleness-aware aggregation to stabilize heterogeneous local workloads. We prove the convergence for non-convex objectives at rate O(1/√R) under bounded staleness, and show that the admission controls yield bounded staleness with high probability under queue-prediction error. Real-world cross-facility deployment of FedQueue shows 20.5% improvement over baseline algorithms. Controlled queue simulations demonstrate robust improvement over the baselines; in particular, up to 60% reduction in time to reach a target accuracy level under high queue variance and non-IID partitions.