# Leakage of Dataset Properties in Multi-Party Machine Learning

arXiv: https://arxiv.org/abs/2006.07267 · HTML: https://mlsanity.com/paper/2006.07267/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 47)

- [Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges](https://mlsanity.com/paper/2201.08135/) (2022)
- [FedFed: Feature Distillation against Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2310.05077/) (2023)
- [SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning](https://mlsanity.com/paper/2212.10986/) (2022)
- [Formalizing and Estimating Distribution Inference Risks](https://mlsanity.com/paper/2109.06024/) (2021)
- [SoK: Memorization in General-Purpose Large Language Models](https://mlsanity.com/paper/2310.18362/) (2023)
- [SNAP: Efficient Extraction of Private Properties with Poisoning](https://mlsanity.com/paper/2208.12348/) (2022)
- [A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy](https://mlsanity.com/paper/2302.10637/) (2023)
- [Group Property Inference Attacks Against Graph Neural Networks](https://mlsanity.com/paper/2209.01100/) (2022)
- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [Distribution inference risks: Identifying and mitigating sources of leakage](https://mlsanity.com/paper/2209.08541/) (2022)
- [Inferring Sensitive Attributes from Model Explanations](https://mlsanity.com/paper/2208.09967/) (2022)
- [Dissecting Distribution Inference](https://mlsanity.com/paper/2212.07591/) (2022)
- [Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges](https://mlsanity.com/paper/2407.06754/) (2024)
- [Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks](https://mlsanity.com/paper/2403.02116/) (2024)
- [Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?](https://mlsanity.com/paper/2407.16607/) (2024)
- [Protecting Global Properties of Datasets with Distribution Privacy Mechanisms](https://mlsanity.com/paper/2207.08367/) (2022)
- [Privacy Leakage in Text Classification: A Data Extraction Approach](https://mlsanity.com/paper/2206.04591/) (2022)
- [Manipulating Transfer Learning for Property Inference](https://mlsanity.com/paper/2303.11643/) (2023)
- [Detecting Data Contamination in LLMs via In-Context Learning](https://mlsanity.com/paper/2510.27055/) (2025)
- [Correlation inference attacks against machine learning models](https://mlsanity.com/paper/2112.08806/) (2021)
- [Combing for Credentials: Active Pattern Extraction from Smart Reply](https://mlsanity.com/paper/2207.10802/) (2022)
- [Summary Statistic Privacy in Data Sharing](https://mlsanity.com/paper/2303.02014/) (2023)
- [Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models](https://mlsanity.com/paper/2306.13789/) (2023)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasures](https://mlsanity.com/paper/2404.00673/) (2024)
- [SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning](https://mlsanity.com/paper/2205.09986/) (2022)
- [Attesting Distributional Properties of Training Data for Machine Learning](https://mlsanity.com/paper/2308.09552/) (2023)
- [Privacy-preserving machine learning for healthcare: open challenges and future perspectives](https://mlsanity.com/paper/2303.15563/) (2023)
- [Can We Infer Confidential Properties of Training Data from LLMs?](https://mlsanity.com/paper/2506.10364/) (2025)
- [Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts](https://mlsanity.com/paper/2510.13451/) (2025)

## Abstract (arXiv)

Secure multi-party machine learning allows several parties to build a model on their pooled data to increase utility while not explicitly sharing data with each other. We show that such multi-party computation can cause leakage of global dataset properties between the parties even when parties obtain only black-box access to the final model. In particular, a ``curious'' party can infer the distribution of sensitive attributes in other parties' data with high accuracy. This raises concerns regarding the confidentiality of properties pertaining to the whole dataset as opposed to individual data records. We show that our attack can leak population-level properties in datasets of different types, including tabular, text, and graph data. To understand and measure the source of leakage, we consider several models of correlation between a sensitive attribute and the rest of the data. Using multiple machine learning models, we show that leakage occurs even if the sensitive attribute is not included in the training data and has a low correlation with other attributes or the target variable.