mlsanityData to 7 Oct 2026

Neural Fine-Tuning Search for Few-Shot Learning

arXiv 2306.09295 ↗First posted 15 Jun 2023cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In few-shot recognition, a classifier that has been trained on one set of classes is required to rapidly adapt and generalize to a disjoint, novel set of classes. To that end, recent studies have shown the efficacy of fine-tuning with carefully crafted adaptation architectures. However this raises the question of: How can one design the optimal adaptation strategy? In this paper, we study this question through the lens of neural architecture search (NAS). Given a pre-trained neural network, our algorithm discovers the optimal arrangement of adapters, which layers to keep frozen and which to fine-tune. We demonstrate the generality of our NAS method by applying it to both residual networks and vision transformers and report state-of-the-art performance on Meta-Dataset and Meta-Album.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 55 of its 67 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

55 of 67 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  11. 2022Visual Prompt Tuningcited by 1,674
  12. 2020Meta-Learning in Neural Networks: A Surveycited by 944
38 more
  1. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  2. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  3. 2017Learning multiple visual domains with residual adapterscited by 553
  4. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  5. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  6. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  7. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  8. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  9. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  10. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  11. 2019A Baseline for Few-Shot Image Classificationcited by 298
  12. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  13. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  14. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  15. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
  16. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  17. 2018SpotTune: Transfer Learning through Adaptive Fine-tuningcited by 166
  18. 2022Neural Prompt Searchcited by 133
  19. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  20. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
  21. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
  22. 2019Improved Few-Shot Visual Classificationcited by 114
  23. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  24. 2019Blockwisely Supervised Neural Architecture Search with Knowledge Distillationcited by 81
  25. 2021Cross-domain Few-shot Learning with Task-specific Adapterscited by 79
  26. 2019Meta-Learning of Neural Architectures for Few-Shot Learningcited by 73
  27. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  28. 2019On Network Design Spaces for Visual Recognitioncited by 68
  29. 2017Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithmcited by 65
  30. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  31. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  32. 2021Universal Representation Learning from Multiple Domains for Few-shot Classificationcited by 56
  33. 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
  34. 2020Enhancing Few-Shot Image Classification with Unlabelled Examplescited by 32
  35. 2020Distilling Optimal Neural Networks: Rapid Search in Diverse Spacescited by 32
  36. 2023Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classificationcited by 29
  37. 2020Optimized Generic Feature Learning for Few-shot Classification across Domainscited by 23
  38. 2020Evolving Search Space for Neural Architecture Searchcited by 23

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2024Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Expertscited by 3
  2. 2025Few-Shot Learning in Video and 3D Object Detection: A Surveycited by 2
  3. 2025Learning Model Successorscited by 0
  4. 2025XTransfer: Modality-Agnostic Few-Shot Model Transfer for Human Sensing at the Edgecited by 0