# Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning

arXiv: https://arxiv.org/abs/2410.07738 · HTML: https://mlsanity.com/paper/2410.07738/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://mlsanity.com/paper/2105.00243/) (2021)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [FedBABU: Towards Enhanced Representation for Federated Image Classification](https://mlsanity.com/paper/2106.06042/) (2021)
- [FedALA: Adaptive Local Aggregation for Personalized Federated Learning](https://mlsanity.com/paper/2212.01197/) (2022)
- [Preservation of the Global Knowledge by Not-True Distillation in Federated Learning](https://mlsanity.com/paper/2106.03097/) (2021)
- [Tackling Data Heterogeneity in Federated Learning with Class Prototypes](https://mlsanity.com/paper/2212.02758/) (2022)
- [FedGH: Heterogeneous Federated Learning with Generalized Global Header](https://mlsanity.com/paper/2303.13137/) (2023)
- [Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning](https://mlsanity.com/paper/2110.08394/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [Eliminating Domain Bias for Federated Learning in Representation Space](https://mlsanity.com/paper/2311.14975/) (2023)
- [FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy](https://mlsanity.com/paper/2307.01217/) (2023)
- [Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning](https://mlsanity.com/paper/2211.07864/) (2022)
- [PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmark](https://mlsanity.com/paper/2312.04992/) (2023)
- [FedREP: Towards Horizontal Federated Load Forecasting for Retail Energy Providers](https://mlsanity.com/paper/2203.00219/) (2022)

## Built on by (most cited first, 6 of 6)

- [A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning](https://mlsanity.com/paper/2504.21099/) (2025)
- [Mitigating Domain Shift in Federated Learning via Intra- and Inter-Domain Prototypes](https://mlsanity.com/paper/2501.08521/) (2025)
- [Enhancing Federated Class-Incremental Learning via Spatial-Temporal Statistics Aggregation](https://mlsanity.com/paper/2506.01327/) (2025)
- [TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models](https://mlsanity.com/paper/2511.16423/) (2025)
- [Gains: Fine-grained Federated Domain Adaptation in Open Set](https://mlsanity.com/paper/2510.15967/) (2025)
- [QoS-Aware Token Scheduling and Private Data Valuation for Multi-Modal Agentic Networks](https://mlsanity.com/paper/2606.15573/) (2026)

## Abstract (arXiv)

Federated Domain Adaptation (FDA) is a Federated Learning (FL) scenario where models are trained across multiple clients with unique data domains but a shared category space, without transmitting private data. The primary challenge in FDA is data heterogeneity, which causes significant divergences in gradient updates when using conventional averaging-based aggregation methods, reducing the efficacy of the global model. This further undermines both in-domain and out-of-domain performance (within the same federated system but outside the local client). To address this, we propose a novel framework called Multi-domain Prototype-based Federated Fine-Tuning (MPFT). MPFT fine-tunes a pre-trained model using multi-domain prototypes, i.e., pretrained representations enriched with domain-specific information from category-specific local data. This enables supervised learning on the server to derive a globally optimized adapter that is subsequently distributed to local clients, without the intrusion of data privacy. Empirical results show that MPFT significantly improves both in-domain and out-of-domain accuracy over conventional methods, enhancing knowledge preservation and adaptation in FDA. Notably, MPFT achieves convergence within a single communication round, greatly reducing computation and communication costs. To ensure privacy, MPFT applies differential privacy to protect the prototypes. Additionally, we develop a prototype-based feature space hijacking attack to evaluate robustness, confirming that raw data samples remain unrecoverable even after extensive training epochs. The complete implementation of MPFL is available at urlhttps://anonymous.4open.science/r/DomainFL/.