mlsanityData to 14 Oct 2026

One to Transfer All: A Universal Transfer Framework for Vision Foundation Model with Few Data

arXiv 2111.12386 ↗First posted 24 Nov 2021cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The foundation model is not the last chapter of the model production pipeline. Transferring with few data in a general way to thousands of downstream tasks is becoming a trend of the foundation model's application. In this paper, we proposed a universal transfer framework: One to Transfer All (OTA) to transfer any Vision Foundation Model (VFM) to any downstream tasks with few downstream data. We first transfer a VFM to a task-specific model by Image Re-representation Fine-tuning (IRF) then distilling knowledge from a task-specific model to a deployed model with data produced by Downstream Image-Guided Generation (DIGG). OTA has no dependency on upstream data, VFM, and downstream tasks when transferring. It also provides a way for VFM researchers to release their upstream information for better transferring but not leaking data due to privacy requirements. Massive experiments validate the effectiveness and superiority of our methods in few data setting. Our code will be released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 52 of its 71 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

52 of 71 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  11. 2017Mask R-CNNcited by 7,277
  12. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
38 more
  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  7. 2013Describing Textures in the Wildcited by 2,125
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  9. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  11. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
  12. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
  13. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  14. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  15. 2019Contrastive Representation Distillationcited by 748
  16. 2021GPT Understands, Toocited by 716
  17. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  18. 2015Optimal Transport for Domain Adaptationcited by 463
  19. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  20. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  21. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 302
  22. 2019Correlation Congruence for Knowledge Distillationcited by 285
  23. 2020Universal Source-Free Domain Adaptationcited by 220
  24. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  25. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  26. 2018SpotTune: Transfer Learning through Adaptive Fine-tuningcited by 168
  27. 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
  28. 2018Importance Weighted Adversarial Nets for Partial Domain Adaptationcited by 90
  29. 2017Partial Transfer Learning with Selective Adversarial Networkscited by 85
  30. 2021WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Trainingcited by 84
  31. 2019DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networkscited by 80
  32. 2021Unsupervised Multi-source Domain Adaptation Without Access to Source Datacited by 76
  33. 2020DMCP: Differentiable Markov Channel Pruning for Neural Networkscited by 69
  34. 2020KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillationcited by 50
  35. 2016Unsupervised Pretraining for Sequence to Sequence Learningcited by 47
  36. 2020Self-supervised Knowledge Distillation for Few-shot Learningcited by 41
  37. 2019An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Modelscited by 39
  38. 2019AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learningcited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.