# Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge

arXiv: https://arxiv.org/abs/2208.13092 · HTML: https://mlsanity.com/paper/2208.13092/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 27 Aug 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Rigging the Lottery: Making All Tickets Winners](https://mlsanity.com/paper/1911.11134/) (2019)
- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science](https://mlsanity.com/paper/1707.04780/) (2017)
- [What's Hidden in a Randomly Weighted Neural Network?](https://mlsanity.com/paper/1911.13299/) (2019)
- [Model Pruning Enables Efficient Federated Learning on Edge Devices](https://mlsanity.com/paper/1909.12326/) (2019)
- [Sparse Networks from Scratch: Faster Training without Losing Performance](https://mlsanity.com/paper/1907.04840/) (2019)
- [FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout](https://mlsanity.com/paper/2102.13451/) (2021)
- [Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch](https://mlsanity.com/paper/2102.04010/) (2021)
- [Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks](https://mlsanity.com/paper/1909.11957/) (2019)
- [Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach](https://mlsanity.com/paper/2001.04756/) (2020)
- [MARINA: Faster Non-Convex Distributed Learning with Compression](https://mlsanity.com/paper/2102.07845/) (2021)
- [Global Sparse Momentum SGD for Pruning Very Deep Neural Networks](https://mlsanity.com/paper/1909.12778/) (2019)
- [LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets](https://mlsanity.com/paper/2008.03371/) (2020)
- [Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better](https://mlsanity.com/paper/2112.09824/) (2021)
- [Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware](https://mlsanity.com/paper/1901.09181/) (2019)
- [Sparse Weight Activation Training](https://mlsanity.com/paper/2001.01969/) (2020)
- [ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity](https://mlsanity.com/paper/2208.02507/) (2022)
- [A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs](https://mlsanity.com/paper/2011.03083/) (2020)
- [Achieving Personalized Federated Learning with Sparse Local Models](https://mlsanity.com/paper/2201.11380/) (2022)

## Built on by (most cited first, 12 of 12)

- [A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks](https://mlsanity.com/paper/2311.07784/) (2023)
- [Federated LoRA with Sparse Communication](https://mlsanity.com/paper/2406.05233/) (2024)
- [Federated Model Heterogeneous Matryoshka Representation Learning](https://mlsanity.com/paper/2406.00488/) (2024)
- [Synergizing Foundation Models and Federated Learning: A Survey](https://mlsanity.com/paper/2406.12844/) (2024)
- [SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead](https://mlsanity.com/paper/2406.00431/) (2024)
- [FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models](https://mlsanity.com/paper/2403.09904/) (2024)
- [All Rivers Run to the Sea: Private Learning with Asymmetric Flows](https://mlsanity.com/paper/2312.05264/) (2023)
- [SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learning](https://mlsanity.com/paper/2405.09037/) (2024)
- [Federated Classification in Hyperbolic Spaces via Secure Aggregation of Convex Hulls](https://mlsanity.com/paper/2308.06895/) (2023)
- [Leveraging Label Potential for Enhanced Multimodal Emotion Recognition](https://mlsanity.com/paper/2504.05158/) (2025)
- [Assortment of Attention Heads: Accelerating Federated PEFT with Head Pruning and Strategic Client Selection](https://mlsanity.com/paper/2506.00743/) (2025)
- [Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity](https://mlsanity.com/paper/2506.00932/) (2025)

## Abstract (arXiv)

Edge devices can benefit remarkably from federated learning due to their distributed nature; however, their limited resource and computing power poses limitations in deployment. A possible solution to this problem is to utilize off-the-shelf sparse learning algorithms at the clients to meet their resource budget. However, such naive deployment in the clients causes significant accuracy degradation, especially for highly resource-constrained clients. In particular, our investigations reveal that the lack of consensus in the sparsity masks among the clients may potentially slow down the convergence of the global model and cause a substantial accuracy drop. With these observations, we present textitfederated lottery aware sparsity hunting (FLASH), a unified sparse learning framework for training a sparse sub-model that maintains the performance under ultra-low parameter density while yielding proportional communication benefits. Moreover, given that different clients may have different resource budgets, we present textithetero-FLASH where clients can take different density budgets based on their device resource limitations instead of supporting only one target parameter density. Experimental analysis on diverse models and datasets shows the superiority of FLASH in closing the gap with an unpruned baseline while yielding up to mathord~10.1% improved accuracy with mathord~10.26× fewer communication, compared to existing alternatives, at similar hyperparameter settings. Code is available at urlhttps://github.com/SaraBabakN/flashfl.