# Lessons Learned: Defending Against Property Inference Attacks

arXiv: https://arxiv.org/abs/2205.08821 · HTML: https://mlsanity.com/paper/2205.08821/

## Key facts

- Cited by 1 papers in the map (data through 7 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 18 May 2022.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 24 of 24)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [Machine Learning Models that Remember Too Much](https://mlsanity.com/paper/1709.07886/) (2017)
- [A Survey of Privacy Attacks in Machine Learning](https://mlsanity.com/paper/2007.07646/) (2020)
- [Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers](https://mlsanity.com/paper/1306.4447/) (2013)
- [ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2102.02551/) (2021)
- [Overlearning Reveals Sensitive Attributes](https://mlsanity.com/paper/1905.11742/) (2019)
- [Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture](https://mlsanity.com/paper/2110.08324/) (2021)
- [Property Inference From Poisoning](https://mlsanity.com/paper/2101.11073/) (2021)
- [DeBayes: a Bayesian Method for Debiasing Network Embeddings](https://mlsanity.com/paper/2002.11442/) (2020)
- [Leakage of Dataset Properties in Multi-Party Machine Learning](https://mlsanity.com/paper/2006.07267/) (2020)
- [Formalizing and Estimating Distribution Inference Risks](https://mlsanity.com/paper/2109.06024/) (2021)

## Built on by (most cited first, 1 of 1)

- [Dissecting Distribution Inference](https://mlsanity.com/paper/2212.07591/) (2022)

## Abstract (arXiv)

This work investigates and evaluates multiple defense strategies against property inference attacks (PIAs), a privacy attack against machine learning models. Given a trained machine learning model, PIAs aim to extract statistical properties of its underlying training data, e.g., reveal the ratio of men and women in a medical training data set. While for other privacy attacks like membership inference, a lot of research on defense mechanisms has been published, this is the first work focusing on defending against PIAs. With the primary goal of developing a generic mitigation strategy against white-box PIAs, we propose the novel approach property unlearning. Extensive experiments with property unlearning show that while it is very effective when defending target models against specific adversaries, property unlearning is not able to generalize, i.e., protect against a whole class of PIAs. To investigate the reasons behind this limitation, we present the results of experiments with the explainable AI tool LIME. They show how state-of-the-art property inference adversaries with the same objective focus on different parts of the target model. We further elaborate on this with a follow-up experiment, in which we use the visualization technique t-SNE to exhibit how severely statistical training data properties are manifested in machine learning models. Based on this, we develop the conjecture that post-training techniques like property unlearning might not suffice to provide the desirable generic protection against PIAs. As an alternative, we investigate the effects of simpler training data preprocessing methods like adding Gaussian noise to images of a training data set on the success rate of PIAs. We conclude with a discussion of the different defense approaches, summarize the lessons learned and provide directions for future work.