# FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification

arXiv: https://arxiv.org/abs/2206.08671 · HTML: https://mlsanity.com/paper/2206.08671/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 77 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)

## Built on by (most cited first, 26 of 26)

- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models](https://mlsanity.com/paper/2401.06432/) (2024)
- [Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation](https://mlsanity.com/paper/2308.15367/) (2023)
- [A Closer Look at Few-shot Classification Again](https://mlsanity.com/paper/2301.12246/) (2023)
- [First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning](https://mlsanity.com/paper/2303.13199/) (2023)
- [Strong Baselines for Parameter Efficient Few-Shot Fine-tuning](https://mlsanity.com/paper/2304.01917/) (2023)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees](https://mlsanity.com/paper/2302.06637/) (2023)
- [Guiding The Last Layer in Federated Learning with Pre-Trained Models](https://mlsanity.com/paper/2306.03937/) (2023)
- [On the Efficacy of Differentially Private Few-shot Image Classification](https://mlsanity.com/paper/2302.01190/) (2023)
- [Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning](https://mlsanity.com/paper/2402.06674/) (2024)
- [Region-Conditioned Orthogonal 3D U-Net for Weather4Cast Competition](https://mlsanity.com/paper/2212.02059/) (2022)
- [Covariances for Free: Exploiting Mean Distributions for Training-free Federated Learning](https://mlsanity.com/paper/2412.14326/) (2024)
- [Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation](https://mlsanity.com/paper/2306.13554/) (2023)
- [Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts](https://mlsanity.com/paper/2403.08477/) (2024)
- [Gradient Inversion Attacks on Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2506.04453/) (2025)
- [F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning](https://mlsanity.com/paper/2509.23173/) (2025)
- [Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Models](https://mlsanity.com/paper/2512.01405/) (2025)
- [Robust Meta-Representation Learning via Global Label Inference and Classification](https://mlsanity.com/paper/2212.11702/) (2022)
- [Federated Fine-Tuning of Foundation Models via Probabilistic Masking](https://mlsanity.com/paper/2311.17299/) (2023)
- [Hyperparameters in Score-Based Membership Inference Attacks](https://mlsanity.com/paper/2502.06374/) (2025)
- [Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Training](https://mlsanity.com/paper/2405.13445/) (2024)
- [Imperfect Vision Encoders: Efficient and Robust Tuning for Vision-Language Models](https://mlsanity.com/paper/2407.16526/) (2024)
- [DP2FL: Dual Prompt Personalized Federated Learning in Foundation Models](https://mlsanity.com/paper/2504.16357/) (2025)
- [Few-shot Classification as Multi-instance Verification: Effective Backbone-agnostic Transfer across Domains](https://mlsanity.com/paper/2507.00401/) (2025)
- [Empirical Comparison of Membership Inference Attacks in Deep Transfer Learning](https://mlsanity.com/paper/2510.05753/) (2025)

## Abstract (arXiv)

Modern deep learning systems are increasingly deployed in situations such as personalization and federated learning where it is necessary to support i) learning on small amounts of data, and ii) communication efficient distributed training protocols. In this work, we develop FiLM Transfer (FiT) which fulfills these requirements in the image classification setting by combining ideas from transfer learning (fixed pretrained backbones and fine-tuned FiLM adapter layers) and meta-learning (automatically configured Naive Bayes classifiers and episodic training) to yield parameter efficient models with superior classification accuracy at low-shot. The resulting parameter efficiency is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. We experiment with FiT on a wide range of downstream datasets and show that it achieves better classification accuracy than the leading Big Transfer (BiT) algorithm at low-shot and achieves state-of-the art accuracy on the challenging VTAB-1k benchmark, with fewer than 1% of the updateable parameters. Finally, we demonstrate the parameter efficiency and superior accuracy of FiT in distributed low-shot applications including model personalization and federated learning where model update size is an important performance metric.