# FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout

arXiv: https://arxiv.org/abs/2307.02623 · HTML: https://mlsanity.com/paper/2307.02623/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 23 of 23)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Model Pruning Enables Efficient Federated Learning on Edge Devices](https://mlsanity.com/paper/1909.12326/) (2019)
- [Gradient Sparsification for Communication-Efficient Distributed Optimization](https://mlsanity.com/paper/1710.09854/) (2017)
- [FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout](https://mlsanity.com/paper/2102.13451/) (2021)
- [Expanding the Reach of Federated Learning by Reducing Client Resource Requirements](https://mlsanity.com/paper/1812.07210/) (2018)
- [Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach](https://mlsanity.com/paper/2001.04756/) (2020)
- [FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers](https://mlsanity.com/paper/2010.05958/) (2020)
- [FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning](https://mlsanity.com/paper/2107.04271/) (2021)
- [CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning](https://mlsanity.com/paper/2112.08909/) (2021)
- [Federated Learning Challenges and Opportunities: An Outlook](https://mlsanity.com/paper/2202.00807/) (2022)
- [FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients](https://mlsanity.com/paper/2201.11865/) (2022)
- [Gap Aware Mitigation of Gradient Staleness](https://mlsanity.com/paper/1909.10802/) (2019)
- [DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems](https://mlsanity.com/paper/2112.08761/) (2021)
- [Taming Momentum in a Distributed Asynchronous Environment](https://mlsanity.com/paper/1907.11612/) (2019)
- [SuperFedNAS: Cost-Efficient Federated Neural Architecture Search for On-Device Inference](https://mlsanity.com/paper/2301.10879/) (2023)
- [FedFly: Towards Migration in Edge-based Distributed Federated Learning](https://mlsanity.com/paper/2111.01516/) (2021)

## Built on by (most cited first, 16 of 16)

- [Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference](https://mlsanity.com/paper/2403.09054/) (2024)
- [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024)
- [CATransformers: Carbon Aware Transformers Through Joint Model-Hardware Optimization](https://mlsanity.com/paper/2505.01386/) (2025)
- [Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach](https://mlsanity.com/paper/2510.21155/) (2025)
- [Characterizing the Efficiency of Distributed Training: A Power, Performance, and Thermal Perspective](https://mlsanity.com/paper/2509.10371/) (2025)
- [Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates](https://mlsanity.com/paper/2403.18375/) (2024)
- [FedTrans: Efficient Federated Learning via Multi-Model Transformation](https://mlsanity.com/paper/2404.13515/) (2024)
- [Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation](https://mlsanity.com/paper/2502.17615/) (2025)
- [Efficient AllReduce with Stragglers](https://mlsanity.com/paper/2505.23523/) (2025)
- [EarthSight: A Distributed Framework for Low-Latency Satellite Intelligence](https://mlsanity.com/paper/2511.10834/) (2025)
- [CLIP: Client-Side Invariant Pruning for Mitigating Stragglers in Secure Federated Learning](https://mlsanity.com/paper/2510.16694/) (2025)
- [FedPSA: Modeling Behavioral Staleness in Asynchronous Federated Learning](https://mlsanity.com/paper/2602.15337/) (2026)
- [Revealing the influence of participant failures on model quality in cross-silo Federated Learning](https://mlsanity.com/paper/2603.25289/) (2026)
- [Robust Synchronisation for Federated Learning in The Face of Correlated Device Failure](https://mlsanity.com/paper/2604.16090/) (2026)
- [OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization](https://mlsanity.com/paper/2605.20276/) (2026)
- [From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning](https://mlsanity.com/paper/2606.10595/) (2026)

## Abstract (arXiv)

Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, straggler devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach.