# Robust Knowledge Distillation in Federated Learning: Counteracting Backdoor Attacks

arXiv: https://arxiv.org/abs/2502.00587 · HTML: https://mlsanity.com/paper/2502.00587/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 1 Feb 2025.

## Placed in

- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping](https://mlsanity.com/paper/2012.13995/) (2020)
- [Attack of the Tails: Yes, You Really Can Backdoor Federated Learning](https://mlsanity.com/paper/2007.05084/) (2020)
- [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://mlsanity.com/paper/2009.01974/) (2020)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [Defending against Backdoors in Federated Learning with Robust Learning Rate](https://mlsanity.com/paper/2007.03767/) (2020)
- [BaFFLe: Backdoor detection via Feedback-based Federated Learning](https://mlsanity.com/paper/2011.02167/) (2020)
- [On the Vulnerability of Backdoor Defenses for Federated Learning](https://mlsanity.com/paper/2301.08170/) (2023)
- [FedRAD: Federated Robust Adaptive Distillation](https://mlsanity.com/paper/2112.01405/) (2021)

## Built on by (most cited first, 2 of 2)

- [FL-PBM: Pre-Training Backdoor Mitigation for Federated Learning](https://mlsanity.com/paper/2603.28673/) (2026)
- [SCRUB-FL: Sanitizing and Cleansing Representations via Unlearning of Backdoors](https://mlsanity.com/paper/2606.22700/) (2026)

## Abstract (arXiv)

Federated Learning (FL) enables collaborative model training across multiple devices while preserving data privacy. However, it remains susceptible to backdoor attacks, where malicious participants can compromise the global model. Existing defence methods are limited by strict assumptions on data heterogeneity (Non-Independent and Identically Distributed data) and the proportion of malicious clients, reducing their practicality and effectiveness. To overcome these limitations, we propose Robust Knowledge Distillation (RKD), a novel defence mechanism that enhances model integrity without relying on restrictive assumptions. RKD integrates clustering and model selection techniques to identify and filter out malicious updates, forming a reliable ensemble of models. It then employs knowledge distillation to transfer the collective insights from this ensemble to a global model. Extensive evaluations demonstrate that RKD effectively mitigates backdoor threats while maintaining high model performance, outperforming current state-of-the-art defence methods across various scenarios.