# Improving Communication Efficiency of Federated Distillation via Accumulating Local Updates

arXiv: https://arxiv.org/abs/2312.04166 · HTML: https://mlsanity.com/paper/2312.04166/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 7 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 6 of 6)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018)
- [Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation](https://mlsanity.com/paper/2204.07028/) (2022)
- [FedICT: Federated Multi-task Distillation for Multi-access Edge Computing](https://mlsanity.com/paper/2301.00389/) (2023)
- [FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence](https://mlsanity.com/paper/2308.07816/) (2023)

## Built on by (most cited first, 1 of 1)

- [Logits Poisoning Attack in Federated Distillation](https://mlsanity.com/paper/2401.03685/) (2024)

## Abstract (arXiv)

As an emerging federated learning paradigm, federated distillation enables communication-efficient model training by transmitting only small-scale knowledge during the learning process. To further improve the communication efficiency of federated distillation, we propose a novel technique, ALU, which accumulates multiple rounds of local updates before transferring the knowledge to the central server. ALU drastically decreases the frequency of communication in federated distillation, thereby significantly reducing the communication overhead during the training process. Empirical experiments demonstrate the substantial effect of ALU in improving the communication efficiency of federated distillation.