mlsanityData to 7 Oct 2026

EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models

arXiv 2608.08138 ↗First posted 8 Aug 2026cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent data protection laws have accelerated the adoption of Federated Learning (FL) for privacy-preserving decentralized training. Nevertheless, increasing model sizes impose substantial computational demands on client devices, limiting FL applicability in resource-constrained settings. We introduce a novel multi-domain federated learning framework in which lightweight client-side proxy models collaborate with a server-side Foundation Model (FM) to learn new concepts without sharing private data. Our approach, EFFEKT, enables efficient server-side training of domain-specific LoRA adapters while preserving feature-space alignment between the FM and proxy extractors via novel bi-directional cross-distillation strategies. Experiments on multiple real-world datasets and deployments on low-power edge devices demonstrate improvements over state-of-the-art baselines in most considered domains while maintaining lightweight computation at the client side.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

41 of 56 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  11. 2016Deep Learning with Differential Privacycited by 2,682
  12. 2019Searching for MobileNetV3cited by 1,984
29 more
  1. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  2. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  3. 2021Model-Contrastive Federated Learningcited by 762
  4. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  5. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  6. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  7. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  8. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  9. 2015A Large-Scale Car Dataset for Fine-Grained Categorization and Verificationcited by 129
  10. 2015Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networkscited by 94
  11. 2023SLoRA: Federated Parameter Efficient Fine-Tuning of Language Modelscited by 92
  12. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  13. 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
  14. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  15. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  16. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  17. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
  18. 2022Continual Learning with Foundation Models: An Empirical Study of Latent Replaycited by 61
  19. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
  20. 2022Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learningcited by 35
  21. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  22. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  23. 2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 28
  24. 2022Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillationcited by 22
  25. 2023Federated Learning for Computer Visioncited by 22
  26. 2024FedFMS: Exploring Federated Foundation Models for Medical Image Segmentationcited by 21
  27. 2024Cross-Architecture Auxiliary Feature Space Translation for Efficient Few-Shot Personalized Object Detectioncited by 5
  28. 2024Theoretical Analysis of Privacy Leakage in Trustworthy Federated Learning: A Perspective from Linear Algebra and Optimization Theorycited by 2
  29. 2025FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Modelscited by 2

Built on by

Not cited yet.