mlsanityData to 14 Oct 2026

Out-of-Distribution Federated Distillation with Domain-Aware Proxy

arXiv 2608.08525 ↗First posted 9 Aug 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning is a distributed machine learning paradigm that trains a global model by aggregating local clients without sharing private data of each client. Federated Distillation (FD) builds upon this paradigm by leveraging knowledge distillation to exchange soft predictions on proxy data instead of model parameters, enabling more efficient communication and supporting heterogeneous model collaboration. However, FD models trained on In-Distribution data are hardly adapted to Out-of-Distribution (OOD) scenarios. In this paper, we propose a domain-aware proxy selection framework to better adopt proxy data for OOD problems. The experimental results show that the proposed models effectively address the challenges of distribution shifts under OOD with and without proxy data by achieving average 82.9% and 80.6% over existing works on standard benchmarks. The codes and data are released in https://anonymous.4open.science/r/DPS-FD-8596.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 56 references that are in the map. First posted on arXiv 9 Aug 2026.

Builds on

31 of 56 references.

References · 31

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  5. 2020In Search of Lost Domain Generalizationcited by 1,054
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  7. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  8. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  9. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  10. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  11. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  12. 2018Large scale distributed neural network training through online distillationcited by 146
19 more
  1. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  2. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  3. 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillationcited by 45
  4. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  5. 2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 30
  6. 2023Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teachercited by 25
  7. 2023Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?cited by 21
  8. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  9. 2024FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Modelscited by 17
  10. 2024FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detectioncited by 14
  11. 2024Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalizationcited by 12
  12. 2023Breaching FedMD: Image Recovery via Paired-Logits Inversion Attackcited by 7
  13. 2025Federated Learning with Domain Shift Erasercited by 7
  14. 2024FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysiscited by 6
  15. 2022Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learningcited by 4
  16. 2025Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalizationcited by 4
  17. 2024Data-Free Federated Class Incremental Learning with Diffusion-Based Generative Memorycited by 3
  18. 2025Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimationcited by 1
  19. 2025Adaptive Federated Distillation for Multi-Domain Non-IID Textual Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Federated Distillation: A Survey
  2. 2024Training Heterogeneous Client Models using Knowledge Distillation in Serverless Federated Learning
  3. 2023Federated Learning on Non-iid Data via Local and Global Distillation
  4. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions
  5. 2021Towards Model Agnostic Federated Learning Using Knowledge Distillation
  6. 2025Federated Distillation on Edge Devices: Efficient Client-Side Filtering for Non-IID Data
  7. 2025Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data
  8. 2022Knowledge Distillation for Federated Learning: a Practical Guide