# Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data

arXiv: https://arxiv.org/abs/2001.02799 · HTML: https://mlsanity.com/paper/2001.02799/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2020.

## Built on by (most cited first, 27 of 27)

- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [Divergence-aware Federated Self-Supervised Learning](https://mlsanity.com/paper/2204.04385/) (2022)
- [Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types](https://mlsanity.com/paper/2103.13318/) (2021)
- [Transferability Estimation using Bhattacharyya Class Separability](https://mlsanity.com/paper/2111.12780/) (2021)
- [Scalable Diverse Model Selection for Accessible Transfer Learning](https://mlsanity.com/paper/2111.06977/) (2021)
- [Self-Supervised Pre-Training for Transformer-Based Person Re-Identification](https://mlsanity.com/paper/2111.12084/) (2021)
- [How stable are Transferability Metrics evaluations?](https://mlsanity.com/paper/2204.01403/) (2022)
- [GAIA: A Transfer Learning System of Object Detection that Fits Your Needs](https://mlsanity.com/paper/2106.11346/) (2021)
- [Too Large; Data Reduction for Vision-Language Pre-Training](https://mlsanity.com/paper/2305.20087/) (2023)
- [Transferability Metrics for Selecting Source Model Ensembles](https://mlsanity.com/paper/2111.13011/) (2021)
- [tf.data service: A Case for Disaggregating ML Input Data Processing](https://mlsanity.com/paper/2210.14826/) (2022)
- [Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks](https://mlsanity.com/paper/2311.01038/) (2023)
- [Label-Free Model Evaluation with Semi-Structured Dataset Representations](https://mlsanity.com/paper/2112.00694/) (2021)
- [Ranking Models in Unlabeled New Environments](https://mlsanity.com/paper/2108.10310/) (2021)
- [Train/Test-Time Adaptation with Retrieval](https://mlsanity.com/paper/2303.14333/) (2023)
- [Supervised Momentum Contrastive Learning for Few-Shot Classification](https://mlsanity.com/paper/2101.11058/) (2021)
- [DIVA: Dataset Derivative of a Learning Task](https://mlsanity.com/paper/2111.09785/) (2021)
- [Scalable Neural Data Server: A Data Recommender for Transfer Learning](https://mlsanity.com/paper/2206.09386/) (2022)
- [Unsupervised Search for Ethnic Minorities' Medical Segmentation Training Set](https://mlsanity.com/paper/2501.02442/) (2025)
- [Modular Adaptation for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2104.00619/) (2021)
- [SHiFT: An Efficient, Flexible Search Engine for Transfer Learning](https://mlsanity.com/paper/2204.01457/) (2022)
- [Outsourcing Training without Uploading Data via Efficient Collaborative Open-Source Sampling](https://mlsanity.com/paper/2210.12575/) (2022)
- [Large-scale Training Data Search for Object Re-identification](https://mlsanity.com/paper/2303.16186/) (2023)
- [TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data](https://mlsanity.com/paper/2111.04798/) (2021)
- [SEPT: Towards Scalable and Efficient Visual Pre-Training](https://mlsanity.com/paper/2212.05473/) (2022)
- [Bipartite Mode Matching for Vision Training Set Search from a Hierarchical Data Server](https://mlsanity.com/paper/2601.09531/) (2026)
- [How to Select Pre-Trained Code Models for Reuse? A Learning Perspective](https://mlsanity.com/paper/2501.03783/) (2025)

## Abstract (arXiv)

Transfer learning has proven to be a successful technique to train deep learning models in the domains where little training data is available. The dominant approach is to pretrain a model on a large generic dataset such as ImageNet and finetune its weights on the target domain. However, in the new era of an ever-increasing number of massive datasets, selecting the relevant data for pretraining is a critical issue. We introduce Neural Data Server (NDS), a large-scale search engine for finding the most useful transfer learning data to the target domain. NDS consists of a dataserver which indexes several large popular image datasets, and aims to recommend data to a client, an end-user with a target application with its own small labeled dataset. The dataserver represents large datasets with a much more compact mixture-of-experts model, and employs it to perform data search in a series of dataserver-client transactions at a low computational cost. We show the effectiveness of NDS in various transfer learning scenarios, demonstrating state-of-the-art performance on several target datasets and tasks such as image classification, object detection and instance segmentation. Neural Data Server is available as a web-service at http://aidemos.cs.toronto.edu/nds/.