# Model Spider: Learning to Rank Pre-Trained Models Efficiently

arXiv: https://arxiv.org/abs/2306.03900 · HTML: https://mlsanity.com/paper/2306.03900/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 120 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 30 of 68)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)

## Built on by (most cited first, 30 of 32)

- [RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs](https://mlsanity.com/paper/2503.10657/) (2025)
- [Selecting Large Language Model to Fine-tune via Rectified Scaling Law](https://mlsanity.com/paper/2402.02314/) (2024)
- [Parrot: Multilingual Visual Instruction Tuning](https://mlsanity.com/paper/2406.02539/) (2024)
- [We Should Chart an Atlas of All the World's Models](https://mlsanity.com/paper/2503.10633/) (2025)
- [Wings: Learning Multimodal LLMs without Text-only Forgetting](https://mlsanity.com/paper/2406.03496/) (2024)
- [RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers](https://mlsanity.com/paper/2510.00202/) (2025)
- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://mlsanity.com/paper/2603.04445/) (2026)
- [Foundation Model is Efficient Multimodal Multitask Model Selector](https://mlsanity.com/paper/2308.06262/) (2023)
- [Which Model to Transfer? A Survey on Transferability Estimation](https://mlsanity.com/paper/2402.15231/) (2024)
- [Efficient Lifelong Model Evaluation in an Era of Rapid Progress](https://mlsanity.com/paper/2402.19472/) (2024)
- [Beimingwu: A Learnware Dock System](https://mlsanity.com/paper/2401.14427/) (2024)
- [INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling](https://mlsanity.com/paper/2505.16303/) (2025)
- [RoboRouter: Training-Free Policy Routing for Robotic Manipulation](https://mlsanity.com/paper/2603.07892/) (2026)
- [Neural Solver Selection for Combinatorial Optimization](https://mlsanity.com/paper/2410.09693/) (2024)
- [Rethinking Model Selection in VLM Through the Lens of Gromov-Wasserstein Distance](https://mlsanity.com/paper/2605.01325/) (2026)
- [Vision-Language Model Selection and Reuse for Downstream Adaptation](https://mlsanity.com/paper/2501.18271/) (2025)
- [SeqFusion: Sequential Fusion of Pre-Trained Models for Zero-Shot Time-Series Forecasting](https://mlsanity.com/paper/2503.02836/) (2025)
- [One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo](https://mlsanity.com/paper/2509.04208/) (2025)
- [OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?](https://mlsanity.com/paper/2410.02735/) (2024)
- [Feature Space Perturbation: A Panacea to Enhanced Transferability Estimation](https://mlsanity.com/paper/2502.16471/) (2025)
- [Estimating Time Series Foundation Model Transferability via In-Context Learning](https://mlsanity.com/paper/2509.23695/) (2025)
- [Pre-Trained Model Recommendation for Downstream Fine-tuning](https://mlsanity.com/paper/2403.06382/) (2024)
- [Match & Choose: Model Selection Framework for Fine-tuning Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.10993/) (2025)
- [SwiftTS: A Swift Selection Framework for Time Series Pre-trained Models via Multi-task Meta-Learning](https://mlsanity.com/paper/2510.23051/) (2025)
- [Discovering Hidden Gems in Model Repositories](https://mlsanity.com/paper/2601.22157/) (2026)
- [ModelLens: Finding the Best for Your Task from Myriads of Models](https://mlsanity.com/paper/2605.07075/) (2026)
- [Continual Model Routing in Evolving Model Hubs](https://mlsanity.com/paper/2605.28577/) (2026)
- [Green Runner: A tool for efficient deep learning component selection](https://mlsanity.com/paper/2401.15810/) (2024)
- [Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Ranking](https://mlsanity.com/paper/2409.16198/) (2024)
- [How to Select Pre-Trained Code Models for Reuse? A Learning Perspective](https://mlsanity.com/paper/2501.03783/) (2025)

## Abstract (arXiv)

Figuring out which Pre-Trained Model (PTM) from a model zoo fits the target task is essential to take advantage of plentiful model resources. With the availability of numerous heterogeneous PTMs from diverse fields, efficiently selecting the most suitable PTM is challenging due to the time-consuming costs of carrying out forward or backward passes over all PTMs. In this paper, we propose Model Spider, which tokenizes both PTMs and tasks by summarizing their characteristics into vectors to enable efficient PTM selection. By leveraging the approximated performance of PTMs on a separate set of training tasks, Model Spider learns to construct tokens and measure the fitness score between a model-task pair via their tokens. The ability to rank relevant PTMs higher than others generalizes to new tasks. With the top-ranked PTM candidates, we further learn to enrich task tokens with their PTM-specific semantics to re-rank the PTMs for better selection. Model Spider balances efficiency and selection ability, making PTM selection like a spider preying on a web. Model Spider demonstrates promising performance in various configurations of model zoos.