mlsanityData to 7 Oct 2026

FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout

arXiv 2307.02623 ↗First posted 5 Jul 2023cs.LG cs.DCcited by 16

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, straggler devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 5 Jul 2023.

Builds on

23 of 40 references.

References · 23

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  7. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  8. 2019Asynchronous Federated Optimizationcited by 264
  9. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  10. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 199
  11. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  12. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
11 more
  1. 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 80
  2. 2020FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tierscited by 64
  3. 2021FedAdapt: Adaptive Offloading for IoT Devices in Federated Learningcited by 26
  4. 2021CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learningcited by 20
  5. 2022Federated Learning Challenges and Opportunities: An Outlookcited by 19
  6. 2019Gap Aware Mitigation of Gradient Stalenesscited by 14
  7. 2022FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clientscited by 14
  8. 2021DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systemscited by 10
  9. 2019Taming Momentum in a Distributed Asynchronous Environmentcited by 9
  10. 2023SuperFedNAS: Cost-Efficient Federated Neural Architecture Search for On-Device Inferencecited by 4
  11. 2021FedFly: Towards Migration in Edge-based Distributed Federated Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 5.

16 papers.

Sort:

By area:

  1. 2024Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inferencecited by 124
  2. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  3. 2025CATransformers: Carbon Aware Transformers Through Joint Model-Hardware Optimizationcited by 7
  4. 2025Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approachcited by 7
  5. 2025Characterizing the Efficiency of Distributed Training: A Power, Performance, and Thermal Perspectivecited by 5
  6. 2024Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updatescited by 4
  7. 2024FedTrans: Efficient Federated Learning via Multi-Model Transformationcited by 1
  8. 2025Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflationcited by 1
  9. 2025Efficient AllReduce with Stragglerscited by 1
  10. 2025EarthSight: A Distributed Framework for Low-Latency Satellite Intelligencecited by 1
  11. 2025CLIP: Client-Side Invariant Pruning for Mitigating Stragglers in Secure Federated Learningcited by 0
  12. 2026FedPSA: Modeling Behavioral Staleness in Asynchronous Federated Learningcited by 0
4 more
  1. 2026Revealing the influence of participant failures on model quality in cross-silo Federated Learningcited by 0
  2. 2026Robust Synchronisation for Federated Learning in The Face of Correlated Device Failurecited by 0
  3. 2026OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularizationcited by 0
  4. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0