# Federated, robust and private distillation

Area: [Knowledge distillation](https://mlsanity.com/distill/index.md). HTML: https://mlsanity.com/distill/federated-robust-and-private-distillation/

## Key facts

- Federated, robust and private distillation: 191 papers on arXiv since 2021 (data through 14 Oct 2026).
- 36 of its 191 mapped papers (19%) cite “Data-Free Knowledge Distillation for Heterogeneous Federated Learning” (2021, arXiv 2105.10056).
- 22 of its 191 mapped papers (12%) cite “FedKD: Communication Efficient Federated Learning via Knowledge Distillation” (2021, arXiv 2108.13323).
- 2 new papers 25 Sep–14 Oct.

## Most cited here

- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021), cited by 36 papers here
- [FedKD: Communication Efficient Federated Learning via Knowledge Distillation](https://mlsanity.com/paper/2108.13323/) (2021), cited by 22 papers here
- [Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning](https://mlsanity.com/paper/2203.09249/) (2022), cited by 20 papers here
- [Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better](https://mlsanity.com/paper/2108.07969/) (2021), cited by 17 papers here
- [Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Data](https://mlsanity.com/paper/2107.00051/) (2021), cited by 15 papers here
- [Reliable Adversarial Distillation with Unreliable Teachers](https://mlsanity.com/paper/2106.04928/) (2021), cited by 14 papers here
- [Knowledge Distillation for Federated Learning: a Practical Guide](https://mlsanity.com/paper/2211.04742/) (2022), cited by 8 papers here
- [FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks](https://mlsanity.com/paper/2201.03169/) (2022), cited by 7 papers here
- [FedFed: Feature Distillation against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2310.05077/) (2023), cited by 7 papers here
- [Towards Model Agnostic Federated Learning Using Knowledge Distillation](https://mlsanity.com/paper/2110.15210/) (2021), cited by 6 papers here

## New (25 Sep–14 Oct)

- [Federated Subspace Guided Vision-Language-Action Policy Distillation for Non-IID Multi-Robot Manipulation](https://mlsanity.com/paper/2609.32239/) (2026-09-26)
- [Unknown-Traffic Detection, Calibration and Shortcut Reliance in Distilled Encrypted-Traffic Classifiers over One Year](https://mlsanity.com/paper/2609.31141/) (2026-09-25)