# Learning Privately from Multiparty Data

arXiv: https://arxiv.org/abs/1602.03552 · HTML: https://mlsanity.com/paper/1602.03552/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2016.

## Built on by (most cited first, 16 of 16)

- [A Study of Face Obfuscation in ImageNet](https://mlsanity.com/paper/2103.06191/) (2021)
- [Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation](https://mlsanity.com/paper/2209.04599/) (2022)
- [Survey: Leakage and Privacy at Inference Time](https://mlsanity.com/paper/2107.01614/) (2021)
- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [An Adaptive Kernel Approach to Federated Learning of Heterogeneous Causal Effects](https://mlsanity.com/paper/2301.00346/) (2023)
- [Practical Differentially Private and Byzantine-resilient Federated Learning](https://mlsanity.com/paper/2304.09762/) (2023)
- [Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks](https://mlsanity.com/paper/2403.02116/) (2024)
- [Frequency Estimation Under Multiparty Differential Privacy: One-shot and Streaming](https://mlsanity.com/paper/2104.01808/) (2021)
- [Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation](https://mlsanity.com/paper/2409.02404/) (2024)
- [The Connection between Out-of-Distribution Generalization and Privacy of ML Models](https://mlsanity.com/paper/2110.03369/) (2021)
- [Inference Attacks: A Taxonomy, Survey, and Promising Directions](https://mlsanity.com/paper/2406.02027/) (2024)
- [Majority Vote for Distributed Differentially Private Sign Selection](https://mlsanity.com/paper/2209.04419/) (2022)
- [Distributed data analytics](https://mlsanity.com/paper/2203.14088/) (2022)
- [Data Analytics with Differential Privacy](https://mlsanity.com/paper/2311.16104/) (2023)
- [Quantum Privacy Aggregation of Teacher Ensembles (QPATE) for Privacy-preserving Quantum Machine Learning](https://mlsanity.com/paper/2401.07464/) (2024)
- [Distributed Deep Variational Approach for Privacy-preserving Data Release](https://mlsanity.com/paper/2605.03069/) (2026)

## Abstract (arXiv)

Learning a classifier from private data collected by multiple parties is an important problem that has many potential applications. How can we build an accurate and differentially private global classifier by combining locally-trained classifiers from different parties, without access to any party's private data? We propose to transfer the `knowledge' of the local classifier ensemble by first creating labeled data from auxiliary unlabeled data, and then train a global ε-differentially private classifier. We show that majority voting is too sensitive and therefore propose a new risk weighted by class probabilities estimated from the ensemble. Relative to a non-private solution, our private solution has a generalization error bounded by O(ε-2M-2) where M is the number of parties. This allows strong privacy without performance loss when M is large, such as in crowdsensing applications. We demonstrate the performance of our method with realistic tasks of activity recognition, network intrusion detection, and malicious URL detection.