# Outsourcing Training without Uploading Data via Efficient Collaborative Open-Source Sampling

arXiv: https://arxiv.org/abs/2210.12575 · HTML: https://mlsanity.com/paper/2210.12575/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 23 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Privacy and Robustness in Federated Learning: Attacks and Defenses](https://mlsanity.com/paper/2012.06337/) (2020)
- [Federated Adversarial Domain Adaptation](https://mlsanity.com/paper/1911.02054/) (2019)
- [In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning](https://mlsanity.com/paper/1809.07857/) (2018)
- [What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation](https://mlsanity.com/paper/2004.11500/) (2020)
- [Limits of Private Learning with Access to Public Data](https://mlsanity.com/paper/1910.11519/) (2019)
- [Leveraging Public Data for Practical Private Query Release](https://mlsanity.com/paper/2102.08598/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization](https://mlsanity.com/paper/2203.09747/) (2022)
- [Deep Private-Feature Extraction](https://mlsanity.com/paper/1802.03151/) (2018)
- [Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling](https://mlsanity.com/paper/2111.01004/) (2021)
- [Positive-Unlabeled Compression on the Cloud](https://mlsanity.com/paper/1909.09757/) (2019)
- [Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data](https://mlsanity.com/paper/2001.02799/) (2020)
- [CalFAT: Calibrated Federated Adversarial Training with Label Skewness](https://mlsanity.com/paper/2205.14926/) (2022)

## Built on by (most cited first, 3 of 3)

- [When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions](https://mlsanity.com/paper/2306.15546/) (2023)
- [Efficient and Effective Model Extraction](https://mlsanity.com/paper/2409.14122/) (2024)
- [The Eminence in Shadow: Exploiting Feature Boundary Ambiguity for Robust Backdoor Attacks](https://mlsanity.com/paper/2512.10402/) (2025)

## Abstract (arXiv)

As deep learning blooms with growing demand for computation and data resources, outsourcing model training to a powerful cloud server becomes an attractive alternative to training at a low-power and cost-effective end device. Traditional outsourcing requires uploading device data to the cloud server, which can be infeasible in many real-world applications due to the often sensitive nature of the collected data and the limited communication bandwidth. To tackle these challenges, we propose to leverage widely available open-source data, which is a massive dataset collected from public and heterogeneous sources (e.g., Internet images). We develop a novel strategy called Efficient Collaborative Open-source Sampling (ECOS) to construct a proximal proxy dataset from open-source data for cloud training, in lieu of client data. ECOS probes open-source data on the cloud server to sense the distribution of client data via a communication- and computation-efficient sampling process, which only communicates a few compressed public features and client scalar responses. Extensive empirical studies show that the proposed ECOS improves the quality of automated client labeling, model compression, and label outsourcing when applied in various learning scenarios.