mlsanityData to 14 Oct 2026

Model Spider: Learning to Rank Pre-Trained Models Efficiently

arXiv 2306.03900 ↗First posted 6 Jun 2023cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Figuring out which Pre-Trained Model (PTM) from a model zoo fits the target task is essential to take advantage of plentiful model resources. With the availability of numerous heterogeneous PTMs from diverse fields, efficiently selecting the most suitable PTM is challenging due to the time-consuming costs of carrying out forward or backward passes over all PTMs. In this paper, we propose Model Spider, which tokenizes both PTMs and tasks by summarizing their characteristics into vectors to enable efficient PTM selection. By leveraging the approximated performance of PTMs on a separate set of training tasks, Model Spider learns to construct tokens and measure the fitness score between a model-task pair via their tokens. The ability to rank relevant PTMs higher than others generalizes to new tasks. With the top-ranked PTM candidates, we further learn to enrich task tokens with their PTM-specific semantics to re-rank the PTMs for better selection. Model Spider balances efficiency and selection ability, making PTM selection like a spider preying on a web. Model Spider demonstrates promising performance in various configurations of model zoos.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 68 of its 120 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

68 of 120 references.

References · 68

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  10. 2016Densely Connected Convolutional Networkscited by 6,720
  11. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  12. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  2. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  4. 2013Describing Textures in the Wildcited by 2,125
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  7. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  8. 2022Visual Prompt Tuningcited by 1,681
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  11. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  12. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  13. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  14. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  15. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  16. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  17. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  18. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  19. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  20. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  21. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  22. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 429
  23. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 347
  24. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
  25. 2018Characterizing and Avoiding Negative Transfercited by 229
  26. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  27. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  28. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 186
  29. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
  30. 2020Geometric Dataset Distances via Optimal Transportcited by 158
  31. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  32. 2019Transferability and Hardness of Supervised Classification Taskscited by 127
  33. 2016HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognitioncited by 104
  34. 2019Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learningcited by 86
  35. 2019Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehensioncited by 81
  36. 2022An Information-Theoretic Approach to Transferability in Task Transfer Learningcited by 81
  37. 2021OTCE: A Transferability Metric for Cross-Domain Cross-Task Representationscited by 62
  38. 2021Transferability Estimation using Bhattacharyya Class Separabilitycited by 54

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 13; 2026: 7.

32 papers.

Sort:

By area:

  1. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 61
  2. 2024Selecting Large Language Model to Fine-tune via Rectified Scaling Lawcited by 33
  3. 2024Parrot: Multilingual Visual Instruction Tuningcited by 30
  4. 2025We Should Chart an Atlas of All the World's Modelscited by 25
  5. 2024Wings: Learning Multimodal LLMs without Text-only Forgettingcited by 23
  6. 2025RouterArena: An Open Platform for Comprehensive Comparison of LLM Routerscited by 22
  7. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 19
  8. 2023Foundation Model is Efficient Multimodal Multitask Model Selectorcited by 16
  9. 2024Which Model to Transfer? A Survey on Transferability Estimationcited by 13
  10. 2024Efficient Lifelong Model Evaluation in an Era of Rapid Progresscited by 10
  11. 2024Beimingwu: A Learnware Dock Systemcited by 8
  12. 2025INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profilingcited by 8
20 more
  1. 2026RoboRouter: Training-Free Policy Routing for Robotic Manipulationcited by 6
  2. 2024Neural Solver Selection for Combinatorial Optimizationcited by 5
  3. 2026Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distancecited by 4
  4. 2025Vision-Language Model Selection and Reuse for Downstream Adaptationcited by 3
  5. 2025SeqFusion: Sequential Fusion of Pre-Trained Models for Zero-Shot Time-Series Forecastingcited by 3
  6. 2025One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoocited by 3
  7. 2024OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?cited by 2
  8. 2025Feature Space Perturbation: A Panacea to Enhanced Transferability Estimationcited by 2
  9. 2025Estimating Time Series Foundation Model Transferability via In-Context Learningcited by 2
  10. 2024Pre-Trained Model Recommendation for Downstream Fine-tuningcited by 1
  11. 2025Match & Choose: Model Selection Framework for Fine-tuning Text-to-Image Diffusion Modelscited by 1
  12. 2025SwiftTS: A Swift Selection Framework for Time Series Pre-trained Models via Multi-task Meta-Learningcited by 1
  13. 2026Discovering Hidden Gems in Model Repositoriescited by 1
  14. 2026ModelLens: Finding the Best for Your Task from Myriads of Modelscited by 1
  15. 2026Continual Model Routing in Evolving Model Hubscited by 1
  16. 2024Green Runner: A tool for efficient deep learning component selectioncited by 0
  17. 2024Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Rankingcited by 0
  18. 2025How to Select Pre-Trained Code Models for Reuse? A Learning Perspectivecited by 0
  19. 2025MetaRank: Task-Aware Metric Selection for Model Transferability Estimationcited by 0
  20. 2026Model Specific Task Similarity for Vision Language Model Selection via Layer Conductancecited by 0