# Parametric Feature Transfer: One-shot Federated Learning with Foundation Models

arXiv: https://arxiv.org/abs/2402.01862 · HTML: https://mlsanity.com/paper/2402.01862/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Feature Generating Networks for Zero-Shot Learning](https://mlsanity.com/paper/1712.00981/) (2017)
- [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://mlsanity.com/paper/2009.01974/) (2020)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [One-Shot Federated Learning](https://mlsanity.com/paper/1902.11175/) (2019)
- [Distilled One-Shot Federated Learning](https://mlsanity.com/paper/2009.07999/) (2020)
- [Heterogeneity for the Win: One-Shot Federated Clustering](https://mlsanity.com/paper/2103.00697/) (2021)
- [When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions](https://mlsanity.com/paper/2306.15546/) (2023)
- [Practical One-Shot Federated Learning for Cross-Silo Setting](https://mlsanity.com/paper/2010.01017/) (2020)
- [A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning](https://mlsanity.com/paper/2210.04428/) (2022)
- [Open-environment Machine Learning](https://mlsanity.com/paper/2206.00423/) (2022)
- [Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space](https://mlsanity.com/paper/2312.09817/) (2023)
- [Cross Domain Generative Augmentation: Domain Generalization with Latent Diffusion Models](https://mlsanity.com/paper/2312.05387/) (2023)
- [Understanding the Role of Layer Normalization in Label-Skewed Federated Learning](https://mlsanity.com/paper/2308.09565/) (2023)

## Built on by (most cited first, 13 of 13)

- [FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models](https://mlsanity.com/paper/2410.04810/) (2024)
- [Vision Foundation Models in Medical Image Analysis: Advances and Challenges](https://mlsanity.com/paper/2502.14584/) (2025)
- [One-shot Federated Learning Methods: A Practical Guide](https://mlsanity.com/paper/2502.09104/) (2025)
- [Capture Global Feature Statistics for One-Shot Federated Learning](https://mlsanity.com/paper/2503.06962/) (2025)
- [Federated Gaussian Mixture Models](https://mlsanity.com/paper/2506.01780/) (2025)
- [One-Shot Federated Learning with Bayesian Pseudocoresets](https://mlsanity.com/paper/2406.02177/) (2024)
- [A New One-Shot Federated Learning Framework for Medical Imaging Classification with Feature-Guided Rectified Flow and Knowledge Distillation](https://mlsanity.com/paper/2507.19045/) (2025)
- [FedLog: Personalized Federated Classification with Less Communication and More Flexibility](https://mlsanity.com/paper/2407.08337/) (2024)
- [One-Shot Federated Unsupervised Domain Adaptation with Scaled Entropy Attention and Multi-Source Smoothed Pseudo Labeling](https://mlsanity.com/paper/2503.10020/) (2025)
- [FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios](https://mlsanity.com/paper/2603.16370/) (2026)
- [Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs](https://mlsanity.com/paper/2605.10748/) (2026)
- [FedOPAL: One-Shot Federated Learning via Analytic Visual Prompt Tuning](https://mlsanity.com/paper/2607.08368/) (2026)
- [CRIP: Channel Level Representation Injection for Personalized One-Shot Federated Learning](https://mlsanity.com/paper/2608.02222/) (2026)

## Abstract (arXiv)

In one-shot federated learning (FL), clients collaboratively train a global model in a single round of communication. Existing approaches for one-shot FL enhance communication efficiency at the expense of diminished accuracy. This paper introduces FedPFT (Federated Learning with Parametric Feature Transfer), a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL. The approach involves transferring per-client parametric models (specifically, Gaussian mixtures) of features extracted from foundation models. Subsequently, each parametric model is employed to generate synthetic features for training a classifier head. Experimental results on eight datasets demonstrate that FedPFT enhances the communication-accuracy frontier in both centralized and decentralized FL scenarios, as well as across diverse data-heterogeneity settings such as covariate shift and task shift, with improvements of up to 20.6%. Additionally, FedPFT adheres to the data minimization principle of FL, as clients do not send real features. We demonstrate that sending real features is vulnerable to potent reconstruction attacks. Moreover, we show that FedPFT is amenable to formal privacy guarantees via differential privacy, demonstrating favourable privacy-accuracy tradeoffs.