mlsanityData to 7 Oct 2026

Ranking Neural Checkpoints

arXiv 2011.11200 ↗First posted 23 Nov 2020cs.LG cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

This paper is concerned with ranking many pre-trained deep neural networks (DNNs), called checkpoints, for the transfer learning to a downstream task. Thanks to the broad use of DNNs, we may easily collect hundreds of checkpoints from various sources. Which of them transfers the best to our downstream task of interest? Striving to answer this question thoroughly, we establish a neural checkpoint ranking benchmark (NeuCRaB) and study some intuitive ranking measures. These measures are generic, applying to the checkpoints of different output types without knowing how the checkpoints are pre-trained on which dataset. They also incur low computation cost, making them practically meaningful. Our results suggest that the linear separability of the features extracted by the checkpoints is a strong indicator of transferability. We also arrive at a new ranking measure, NLEEP, which gives rise to the best performance in the experiments.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 4; 2023: 11; 2024: 8; 2025: 15; 2026: 5.

50 papers.

Sort:

By area:

  1. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 62
  2. 2021Transferability Estimation using Bhattacharyya Class Separabilitycited by 53
  3. 2021Frustratingly Easy Transferability Estimationcited by 44
  4. 2023Model Spider: Learning to Rank Pre-Trained Models Efficientlycited by 32
  5. 2022PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Taskscited by 30
  6. 2022How stable are Transferability Metrics evaluations?cited by 30
  7. 2022Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Spacecited by 30
  8. 2025Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routingcited by 25
  9. 2025We Should Chart an Atlas of All the World's Modelscited by 25
  10. 2021Transferability Metrics for Selecting Source Model Ensemblescited by 24
  11. 2023ETran: Energy-Based Transferability Estimationcited by 18
  12. 2023Foundation Model is Efficient Multimodal Multitask Model Selectorcited by 16
38 more
  1. 2021Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performancecited by 15
  2. 2024Which Model to Transfer? A Survey on Transferability Estimationcited by 13
  3. 2023Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentationcited by 10
  4. 2023Exploring Model Transferability through the Lens of Potential Energycited by 10
  5. 2023The Performance of Transferability Metrics does not Translate to Medical Taskscited by 6
  6. 2024Bounding Box Stability against Feature Dropout Reflects Detector Generalization across Environmentscited by 6
  7. 2024Assessing Pre-Trained Models for Transfer Learning Through Distribution of Spectral Componentscited by 6
  8. 2021Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPscited by 5
  9. 2023On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topologycited by 5
  10. 2023Simple Transferability Estimation for Regression Taskscited by 5
  11. 2025Sequence Transferability and Task Order Selection in Continual Learningcited by 5
  12. 2023How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Surveycited by 4
  13. 2025Exploring Structural Degradation in Dense Representations for Self-supervised Learningcited by 4
  14. 2021Exploiting a Zoo of Checkpoints for Unseen Taskscited by 3
  15. 2025LEAD: Exploring Logit Space Evolution for Model Selectioncited by 3
  16. 2023Transferability Metrics for Object Detectioncited by 2
  17. 2025Feature Space Perturbation: A Panacea to Enhanced Transferability Estimationcited by 2
  18. 2025Benchmarking Transferability: A Framework for Fair and Robust Evaluationcited by 2
  19. 2025Estimating Time Series Foundation Model Transferability via In-Context Learningcited by 2
  20. 2025How NOT to benchmark your SITE metric: Beyond Static Leaderboards and Towards Realistic Evaluationcited by 2
  21. 2022Selective Cross-Task Distillationcited by 1
  22. 2023Unleashing the power of Neural Collapse for Transferability Estimationcited by 1
  23. 2024Pre-Trained Model Recommendation for Downstream Fine-tuningcited by 1
  24. 2025k-NN as a Simple and Effective Estimator of Transferabilitycited by 1
  25. 2025Analysis of Transferability Estimation Metrics for Surgical Phase Recognitioncited by 1
  26. 2026Unlocking Few-Shot Capabilities in LVLMs via Prompt Conditioning and Head Selectioncited by 1
  27. 2026ModelLens: Finding the Best for Your Task from Myriads of Modelscited by 1
  28. 2024Efficient Transferability Assessment for Selection of Pre-trained Detectorscited by 0
  29. 2024Back to the Basics on Predicting Transfer Performancecited by 0
  30. 2024Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Rankingcited by 0
  31. 2024Enhancing pretraining efficiency for medical image segmentation via transferability metricscited by 0
  32. 2025Mordal: Automated Pretrained Model Selection for Vision Language Modelscited by 0
  33. 2025Unsupervised Source-Free Ranking of Biomedical Segmentation Models Under Distribution Shiftcited by 0
  34. 2025Implicit Modeling for Transferability Estimation of Vision Foundation Modelscited by 0
  35. 2025MetaRank: Task-Aware Metric Selection for Model Transferability Estimationcited by 0
  36. 2026Data Distribution Valuation Using Generalized Bayesian Inferencecited by 0
  37. 2026Topology-Driven Transferability Estimation for 3D Medical Vision Foundation Modelscited by 0
  38. 2026Robustness of transferability estimation metrics for medical imagingcited by 0