mlsanityData to 14 Oct 2026

Frozen Feature Augmentation for Few-Shot Image Classification

arXiv 2403.10519 ↗First posted 15 Mar 2024cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Training a linear classifier or lightweight model on top of pretrained vision model outputs, so-called 'frozen features', leads to impressive performance on a number of downstream few-shot tasks. Currently, frozen features are not modified during training. On the other hand, when networks are trained directly on images, data augmentation is a standard recipe that improves performance with no substantial overhead. In this paper, we conduct an extensive pilot study on few-shot image classification that explores applying data augmentations in the frozen feature space, dubbed 'frozen feature augmentation (FroFA)', covering twenty augmentations in total. Our study demonstrates that adopting a deceptively simple pointwise FroFA, such as brightness, can improve few-shot performance consistently across three network architectures, three large pretraining datasets, and eight transfer datasets.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 60 of its 78 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

60 of 78 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
38 more
  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  5. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  6. 2016Matching Networks for One Shot Learningcited by 2,255
  7. 2013Describing Textures in the Wildcited by 2,125
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  10. 2022Visual Prompt Tuningcited by 1,681
  11. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  12. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  13. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  14. 2021Scaling Vision Transformerscited by 999
  15. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  16. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  17. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  18. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  19. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  20. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  21. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  22. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  23. 2021ImageNet-21K Pretraining for the Massescited by 613
  24. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  25. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  26. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  27. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  28. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  29. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 362
  30. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 227
  31. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  32. 2017Dataset Augmentation in Feature Spacecited by 124
  33. 2021Vision Transformer for Small-Size Datasetscited by 122
  34. 2019Improved Few-Shot Visual Classificationcited by 115
  35. 2021Efficient Training of Visual Transformers with Small Datasetscited by 108
  36. 2022Better plain ViT baselines for ImageNet-1kcited by 98
  37. 2016Improving Neural Language Models with a Continuous Cachecited by 82
  38. 2022Parameter-efficient Model Adaptation for Vision Transformerscited by 81

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 7.

13 papers.

Sort:

By area:

  1. 2024Multimodal Whole Slide Foundation Model for Pathologycited by 162
  2. 2026CARE: A Molecular-Guided Foundation Model with Adaptive Region Modeling for Whole Slide Image Analysiscited by 6
  3. 2024No Train, all Gain: Self-Supervised Gradients Improve Deep Frozen Representationscited by 5
  4. 2026Exact Federated Continual Unlearning for Ridge Heads on Frozen Foundation Modelscited by 4
  5. 2025Slot Attention-based Feature Filtering for Few-Shot Learningcited by 1
  6. 2026Prompt-Driven Simulation with Feature Perturbation for Cross-Domain Few-Shot Object Detectioncited by 1
  7. 2025Group Relative Augmentation for Data Efficient Action Detectioncited by 0
  8. 2025Color as the Impetus: Transforming Few-Shot Learnercited by 0
  9. 2025Denoised Diffusion for Object-Focused Image Augmentationcited by 0
  10. 2026ReCCur: A Recursive Corner-Case Curation Framework for Robust Vision-Language Understanding in Open and Edge Scenarioscited by 0
  11. 2026VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfercited by 0
  12. 2026FACT: A Simple and Efficient Framework for Active Finetuningcited by 0
1 more
  1. 2026Whitening Improves Robustness to Spurious Correlations in Linear Probescited by 0