# CaPC Learning: Confidential and Private Collaborative Learning

arXiv: https://arxiv.org/abs/2102.05188 · HTML: https://mlsanity.com/paper/2102.05188/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [InstaHide: Instance-hiding Schemes for Private Distributed Learning](https://mlsanity.com/paper/2010.02772/) (2020)
- [TextHide: Tackling Data Privacy in Language Understanding Tasks](https://mlsanity.com/paper/2010.06053/) (2020)
- [nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data](https://mlsanity.com/paper/1810.10121/) (2018)
- [Privacy-preserving Learning via Deep Net Pruning](https://mlsanity.com/paper/2003.01876/) (2020)

## Built on by (most cited first, 30 of 33)

- [Federated Learning for Smart Healthcare: A Survey](https://mlsanity.com/paper/2111.08834/) (2021)
- [Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models](https://mlsanity.com/paper/2305.15594/) (2023)
- [Teach LLMs to Phish: Stealing Private Information from Language Models](https://mlsanity.com/paper/2403.00871/) (2024)
- [Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2203.09702/) (2022)
- [Disparate Impact in Differential Privacy from Gradient Misalignment](https://mlsanity.com/paper/2206.07737/) (2022)
- [ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training](https://mlsanity.com/paper/2110.05323/) (2021)
- [Practical Differentially Private and Byzantine-resilient Federated Learning](https://mlsanity.com/paper/2304.09762/) (2023)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey](https://mlsanity.com/paper/2505.01821/) (2025)
- [The Future of Consumer Edge-AI Computing](https://mlsanity.com/paper/2210.10514/) (2022)
- [Federated Learning Attacks and Defenses: A Survey](https://mlsanity.com/paper/2211.14952/) (2022)
- [Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives](https://mlsanity.com/paper/2411.05818/) (2024)
- [SoK: Machine Learning Governance](https://mlsanity.com/paper/2109.10870/) (2021)
- [Individualized PATE: Differentially Private Machine Learning with Individual Privacy Guarantees](https://mlsanity.com/paper/2202.10517/) (2022)
- [Private, Efficient, and Accurate: Protecting Models Trained by Multi-party Learning with Differential Privacy](https://mlsanity.com/paper/2208.08662/) (2022)
- [Private and Reliable Neural Network Inference](https://mlsanity.com/paper/2210.15614/) (2022)
- [Certification for Differentially Private Prediction in Gradient-Based Training](https://mlsanity.com/paper/2406.13433/) (2024)
- [SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning](https://mlsanity.com/paper/2205.09986/) (2022)
- [Hierarchical Federated Learning with Privacy](https://mlsanity.com/paper/2206.05209/) (2022)
- [CYCle: Choosing Your Collaborators Wisely to Enhance Collaborative Fairness in Decentralized Learning](https://mlsanity.com/paper/2501.12344/) (2025)
- [In Differential Privacy, There is Truth: On Vote Leakage in Ensemble Private Learning](https://mlsanity.com/paper/2209.10732/) (2022)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [Why Train More? Effective and Efficient Membership Inference via Memorization](https://mlsanity.com/paper/2310.08015/) (2023)
- [Wildest Dreams: Reproducible Research in Privacy-preserving Neural Network Training](https://mlsanity.com/paper/2403.03592/) (2024)
- [FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning](https://mlsanity.com/paper/2410.03042/) (2024)
- [Privacy-Preserving Federated Learning via System Immersion and Random Matrix Encryption](https://mlsanity.com/paper/2204.02497/) (2022)
- [Robust and Actively Secure Serverless Collaborative Learning](https://mlsanity.com/paper/2310.16678/) (2023)
- [Certification-Based Differentially Private Learning](https://mlsanity.com/paper/2609.39629/) (2026)
- [Private Multi-Winner Voting for Machine Learning](https://mlsanity.com/paper/2211.15410/) (2022)
- [Encrypted machine learning of molecular quantum properties](https://mlsanity.com/paper/2212.04322/) (2022)

## Abstract (arXiv)

Machine learning benefits from large training datasets, which may not always be possible to collect by any single entity, especially when using privacy-sensitive data. In many contexts, such as healthcare and finance, separate parties may wish to collaborate and learn from each other's data but are prevented from doing so due to privacy regulations. Some regulations prevent explicit sharing of data between parties by joining datasets in a central location (confidentiality). Others also limit implicit sharing of data, e.g., through model predictions (privacy). There is currently no method that enables machine learning in such a setting, where both confidentiality and privacy need to be preserved, to prevent both explicit and implicit sharing of data. Federated learning only provides confidentiality, not privacy, since gradients shared still contain private information. Differentially private learning assumes unreasonably large datasets. Furthermore, both of these learning paradigms produce a central model whose architecture was previously agreed upon by all parties rather than enabling collaborative learning where each party learns and improves their own local model. We introduce Confidential and Private Collaborative (CaPC) learning, the first method provably achieving both confidentiality and privacy in a collaborative setting. We leverage secure multi-party computation (MPC), homomorphic encryption (HE), and other techniques in combination with privately aggregated teacher models. We demonstrate how CaPC allows participants to collaborate without having to explicitly join their training sets or train a central model. Each party is able to improve the accuracy and fairness of their model, even in settings where each party has a model that performs well on their own dataset or when datasets are not IID and model architectures are heterogeneous across parties.