# FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update

arXiv: https://arxiv.org/abs/2108.09081 · HTML: https://mlsanity.com/paper/2108.09081/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [Structured Pruning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.08571/) (2015)
- [Federated Learning for Emoji Prediction in a Mobile Keyboard](https://mlsanity.com/paper/1906.04329/) (2019)
- [Fast ConvNets Using Group-wise Brain Damage](https://mlsanity.com/paper/1506.02515/) (2015)
- [LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets](https://mlsanity.com/paper/2008.03371/) (2020)
- [A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networks](https://mlsanity.com/paper/1901.04359/) (2019)
- [Distilling Critical Paths in Convolutional Neural Networks](https://mlsanity.com/paper/1811.02643/) (2018)

## Built on by (most cited first, 4 of 4)

- [Tackling Noisy Clients in Federated Learning with End-to-end Label Correction](https://mlsanity.com/paper/2408.04301/) (2024)
- [FedRN: Exploiting k-Reliable Neighbors Towards Robust Federated Learning](https://mlsanity.com/paper/2205.01310/) (2022)
- [FedLesScan: Mitigating Stragglers in Serverless Federated Learning](https://mlsanity.com/paper/2211.05739/) (2022)
- [Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Models](https://mlsanity.com/paper/2503.10218/) (2025)

## Abstract (arXiv)

Federated learning aims to protect users' privacy while performing data analysis from different participants. However, it is challenging to guarantee the training efficiency on heterogeneous systems due to the various computational capabilities and communication bottlenecks. In this work, we propose FedSkel to enable computation-efficient and communication-efficient federated learning on edge devices by only updating the model's essential parts, named skeleton networks. FedSkel is evaluated on real edge devices with imbalanced datasets. Experimental results show that it could achieve up to 5.52× speedups for CONV layers' back-propagation, 1.82× speedups for the whole training process, and reduce 64.8% communication cost, with negligible accuracy loss.