mlsanityData to 14 Oct 2026

Simple Transferability Estimation for Regression Tasks

arXiv 2312.00656 ↗First posted 1 Dec 2023cs.LG cs.AI stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We consider transferability estimation, the problem of estimating how well deep learning models transfer from a source to a target task. We focus on regression tasks, which received little previous attention, and propose two simple and computationally efficient approaches that estimate transferability based on the negative regularized mean squared error of a linear regression model. We prove novel theoretical results connecting our approaches to the actual transferability of the optimal target models obtained from the transfer learning process. Despite their simplicity, our approaches significantly outperform existing state-of-the-art regression transferability estimators in both accuracy and efficiency. On two large-scale keypoint regression benchmarks, our approaches yield 12% to 36% better results on average while being at least 27% faster than previous state-of-the-art methods.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

41 of 60 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2014Going Deeper with Convolutionscited by 4,241
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  9. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  10. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  11. 2018A Survey on Deep Transfer Learningcited by 419
  12. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
29 more
  1. 2019A Baseline for Few-Shot Image Classificationcited by 298
  2. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  3. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  4. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  5. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  6. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  7. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
  8. 2019Transferability and Hardness of Supervised Classification Taskscited by 127
  9. 2014Return of the Devil in the Details: Delving Deep into Convolutional Netscited by 122
  10. 2019Regularized Learning for Domain Adaptation under Label Shiftscited by 120
  11. 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
  12. 2019Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learningcited by 86
  13. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 81
  14. 2019Sim2real transfer learning for 3D human pose estimation: motion to the rescuecited by 76
  15. 2021OTCE: A Transferability Metric for Cross-Domain Cross-Task Representationscited by 62
  16. 2020Estimating Generalization under Distribution Shifts via Domain-Invariant Representationscited by 56
  17. 2020Ranking Neural Checkpointscited by 51
  18. 2021Frustratingly Easy Transferability Estimationcited by 45
  19. 2021A linearized framework and a new benchmark for model selection for fine-tuningcited by 41
  20. 2021Scalable Diverse Model Selection for Accessible Transfer Learningcited by 40
  21. 2014Analyzing the Performance of Multilayer Neural Networks for Object Recognitioncited by 29
  22. 2021GAIA: A Transfer Learning System of Object Detection that Fits Your Needscited by 27
  23. 2020Duality Diagram Similarity: a generic framework for initialization selection in task transfer learningcited by 23
  24. 2019FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learningcited by 10
  25. 2021ASMNet: a Lightweight Deep Neural Network for Face Alignment and Pose Estimationcited by 10
  26. 2022Transfer Learning with Kernel Methodscited by 5
  27. 2022Generalization Bounds for Deep Transfer Learning Using Majority Predictor Accuracycited by 4
  28. 2021Exploiting a Zoo of Checkpoints for Unseen Taskscited by 3
  29. 2020Deep Transfer Learning For Plant Center Localizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Sequence Transferability and Task Order Selection in Continual Learningcited by 5
  2. 2025Understanding Knowledge Transferability for Transfer Learning: A Surveycited by 4
  3. 2025Transfer Learning for Benign Overfitting in High-Dimensional Linear Regressioncited by 3
  4. 2025Estimating Time Series Foundation Model Transferability via In-Context Learningcited by 2
  5. 2026Data Distribution Valuation Using Generalized Bayesian Inferencecited by 0