mlsanityData to 7 Oct 2026

Domain Adaptive Transfer Learning with Specialist Models

arXiv 1811.07056 ↗First posted 16 Nov 2018cs.CV cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Transfer learning is a widely used method to build high performing computer vision models. In this paper, we study the efficacy of transfer learning by examining how the choice of data impacts performance. We find that more pre-training data does not always help, and transfer performance depends on a judicious choice of pre-training data. These findings are important given the continued increase in dataset sizes. We further propose domain adaptive transfer learning, a simple and effective pre-training method using importance weights computed based on the target dataset. Our method to compute importance weights follow from ideas in domain adaptation, and we show a novel application to transfer learning. Our methods achieve state-of-the-art results on multiple fine-grained classification datasets and are well-suited for use in practice.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 14; 2022: 10; 2023: 5; 2024: 2.

31 papers.

Sort:

By area:

  1. 2021ImageNet-21K Pretraining for the Massescited by 607
  2. 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 194
  3. 2021Exploring the Limits of Large Scale Pre-trainingcited by 104
  4. 2021Self-Supervised Pretraining Improves Self-Supervised Pretrainingcited by 81
  5. 2022What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factorscited by 61
  6. 2022Transferability in Deep Learning: A Surveycited by 57
  7. 2023PRIOR: Prototype Representation Joint Learning from Medical Images and Reportscited by 55
  8. 2021Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Typescited by 54
  9. 2021Transferability Estimation using Bhattacharyya Class Separabilitycited by 53
  10. 2024Current state of LLM Risks and AI Guardrailscited by 44
  11. 2021Adaptive Consistency Regularization for Semi-Supervised Transfer Learningcited by 37
  12. 2022A Data-Based Perspective on Transfer Learningcited by 32
19 more
  1. 2022How stable are Transferability Metrics evaluations?cited by 30
  2. 2021Transferability Metrics for Selecting Source Model Ensemblescited by 24
  3. 2021Auxiliary Task Update Decomposition: The Good, The Bad and The Neutralcited by 17
  4. 2022Deep transfer learning for image classification: a surveycited by 13
  5. 2023Adaptive Training Distributions with Scalable Online Bilevel Optimizationcited by 11
  6. 2021Fine-Grained Adversarial Semi-supervised Learningcited by 4
  7. 2022Scalable Neural Data Server: A Data Recommender for Transfer Learningcited by 4
  8. 2022ScaleNet: Searching for the Model to Scalecited by 4
  9. 2023Fairness-Aware Data Valuation for Supervised Learningcited by 4
  10. 2023Towards Understanding the Effect of Pretraining Label Granularitycited by 3
  11. 2024A Bayesian Approach to Data Point Selectioncited by 3
  12. 2021Non-binary deep transfer learning for image classificationcited by 2
  13. 2022SEPT: Towards Scalable and Efficient Visual Pre-Trainingcited by 1
  14. 2023Mixture Data for Training Cannot Ensure Out-of-distribution Generalizationcited by 1
  15. 2021Sequential Random Network for Fine-grained Image Classificationcited by 0
  16. 2021Fighting Selection Bias in Statistical Learning: Application to Visual Recognition from Biased Image Databasescited by 0
  17. 2021Learning from Mistakes based on Class Weighting with Application to Neural Architecture Searchcited by 0
  18. 2022ISTRBoost: Importance Sampling Transfer Regression using Boostingcited by 0
  19. 2022Classification of Melanocytic Nevus Images using BigTransfer (BiT)cited by 0