# Lossy Compression of Noisy Data for Private and Data-Efficient Learning

arXiv: https://arxiv.org/abs/2202.02892 · HTML: https://mlsanity.com/paper/2202.02892/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 54 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [Towards Deep Neural Network Architectures Robust to Adversarial Examples](https://mlsanity.com/paper/1412.5068/) (2014)
- [Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples](https://mlsanity.com/paper/1803.05787/) (2018)
- [Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression](https://mlsanity.com/paper/1705.02900/) (2017)
- [A Study of Face Obfuscation in ImageNet](https://mlsanity.com/paper/2103.06191/) (2021)
- [Defeating Image Obfuscation with Deep Learning](https://mlsanity.com/paper/1609.00408/) (2016)
- [An Information-Theoretic Justification for Model Pruning](https://mlsanity.com/paper/2102.08329/) (2021)
- [Efficient Nonlinear Transforms for Lossy Image Compression](https://mlsanity.com/paper/1802.00847/) (2018)
- [LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks](https://mlsanity.com/paper/2111.08988/) (2021)
- [Neural Distributed Image Compression using Common Information](https://mlsanity.com/paper/2106.11723/) (2021)
- [Information Extraction Under Privacy Constraints](https://mlsanity.com/paper/1511.02381/) (2015)
- [Generative Adversarial Privacy](https://mlsanity.com/paper/1807.05306/) (2018)
- [Protecting JPEG Images Against Adversarial Attacks](https://mlsanity.com/paper/1803.00940/) (2018)
- [LFZip: Lossy compression of multivariate floating-point time series data via improved prediction](https://mlsanity.com/paper/1911.00208/) (2019)

## Built on by (most cited first, 1 of 1)

- [Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation](https://mlsanity.com/paper/2306.04924/) (2023)

## Abstract (arXiv)

Storage-efficient privacy-preserving learning is crucial due to increasing amounts of sensitive user data required for modern learning tasks. We propose a framework for reducing the storage cost of user data while at the same time providing privacy guarantees, without essential loss in the utility of the data for learning. Our method comprises noise injection followed by lossy compression. We show that, when appropriately matching the lossy compression to the distribution of the added noise, the compressed examples converge, in distribution, to that of the noise-free training data as the sample size of the training data (or the dimension of the training data) increases. In this sense, the utility of the data for learning is essentially maintained, while reducing storage and privacy leakage by quantifiable amounts. We present experimental results on the CelebA dataset for gender classification and find that our suggested pipeline delivers in practice on the promise of the theory: the individuals in the images are unrecognizable (or less recognizable, depending on the noise level), overall storage of the data is substantially reduced, with no essential loss (and in some cases a slight boost) to the classification accuracy. As an added bonus, our experiments suggest that our method yields a substantial boost to robustness in the face of adversarial test data.