# SHiFT: An Efficient, Flexible Search Engine for Transfer Learning

arXiv: https://arxiv.org/abs/2204.01457 · HTML: https://mlsanity.com/paper/2204.01457/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 59 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 26 of 26)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Non-stochastic Best Arm Identification and Hyperparameter Optimization](https://mlsanity.com/paper/1502.07943/) (2015)
- [Task2Vec: Task Embedding for Meta-Learning](https://mlsanity.com/paper/1902.03545/) (2019)
- [LEEP: A New Measure to Evaluate Transferability of Learned Representations](https://mlsanity.com/paper/2002.12462/) (2020)
- [Transferability and Hardness of Supervised Classification Tasks](https://mlsanity.com/paper/1908.08142/) (2019)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types](https://mlsanity.com/paper/2103.13318/) (2021)
- [Scalable Transfer Learning with Expert Models](https://mlsanity.com/paper/2009.13239/) (2020)
- [A linearized framework and a new benchmark for model selection for fine-tuning](https://mlsanity.com/paper/2102.00084/) (2021)
- [Adversarial Training Helps Transfer Learning via Better Representations](https://mlsanity.com/paper/2106.10189/) (2021)
- [ZeroER: Entity Resolution using Zero Labeled Examples](https://mlsanity.com/paper/1908.06049/) (2019)
- [Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data](https://mlsanity.com/paper/2001.02799/) (2020)
- [Which Model to Transfer? Finding the Needle in the Growing Haystack](https://mlsanity.com/paper/2010.06402/) (2020)
- [Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads](https://mlsanity.com/paper/1708.07308/) (2017)
- [Complaint-driven Training Data Debugging for Query 2.0](https://mlsanity.com/paper/2004.05722/) (2020)
- [On Convergence of Nearest Neighbor Classifiers over Feature Transformations](https://mlsanity.com/paper/2010.07765/) (2020)
- [Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems](https://mlsanity.com/paper/2108.05053/) (2021)
- [Theoretical Guarantees of Transfer Learning](https://mlsanity.com/paper/1810.05986/) (2018)

## Built on by (most cited first, 3 of 3)

- [Model Selection with Model Zoo via Graph Learning](https://mlsanity.com/paper/2404.03988/) (2024)
- [A Two-Phase Recall-and-Select Framework for Fast Model Selection](https://mlsanity.com/paper/2404.00069/) (2024)
- [Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement](https://mlsanity.com/paper/2512.05525/) (2025)

## Abstract (arXiv)

Transfer learning can be seen as a data- and compute-efficient alternative to training models from scratch. The emergence of rich model repositories, such as TensorFlow Hub, enables practitioners and researchers to unleash the potential of these models across a wide range of downstream tasks. As these repositories keep growing exponentially, efficiently selecting a good model for the task at hand becomes paramount. By carefully comparing various selection and search strategies, we realize that no single method outperforms the others, and hybrid or mixed strategies can be beneficial. Therefore, we propose SHiFT, the first downstream task-aware, flexible, and efficient model search engine for transfer learning. These properties are enabled by a custom query language SHiFT-QL together with a cost-based decision maker, which we empirically validate. Motivated by the iterative nature of machine learning development, we further support efficient incremental executions of our queries, which requires a careful implementation when jointly used with our optimizations.