# Differential Privacy as a Mutual Information Constraint

arXiv: https://arxiv.org/abs/1608.03677 · HTML: https://mlsanity.com/paper/1608.03677/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2016.

## Built on by (most cited first, 30 of 45)

- [Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano](https://mlsanity.com/paper/2210.13662/) (2022)
- [Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy](https://mlsanity.com/paper/2507.06969/) (2025)
- [Stochastic Coded Federated Learning with Convergence and Privacy Guarantees](https://mlsanity.com/paper/2201.10092/) (2022)
- [Simple Binary Hypothesis Testing under Local Differential Privacy and Communication Constraints](https://mlsanity.com/paper/2301.03566/) (2023)
- [Quantum Pufferfish Privacy: A Flexible Privacy Framework for Quantum Systems](https://mlsanity.com/paper/2306.13054/) (2023)
- [Noiseless Privacy-Preserving Decentralized Learning](https://mlsanity.com/paper/2404.09536/) (2024)
- [Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning](https://mlsanity.com/paper/2202.02423/) (2022)
- [Universal Exact Compression of Differentially Private Mechanisms](https://mlsanity.com/paper/2405.20782/) (2024)
- [Contraction of Private Quantum Channels and Private Quantum Hypothesis Testing](https://mlsanity.com/paper/2406.18651/) (2024)
- [Communication efficient privacy-preserving distributed optimization using adaptive differential quantization](https://mlsanity.com/paper/2105.14416/) (2021)
- [Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design](https://mlsanity.com/paper/2211.04132/) (2022)
- [Eliminating Label Leakage in Tree-Based Vertical Federated Learning](https://mlsanity.com/paper/2307.10318/) (2023)
- [Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization](https://mlsanity.com/paper/2407.09324/) (2024)
- [Revisiting Differentially Private Hyper-parameter Tuning](https://mlsanity.com/paper/2402.13087/) (2024)
- [BadGD: A unified data-centric framework to identify gradient descent vulnerabilities](https://mlsanity.com/paper/2405.15979/) (2024)
- [Defending Against Data Reconstruction Attacks in Federated Learning: An Information Theory Approach](https://mlsanity.com/paper/2403.01268/) (2024)
- [Analyzing Inference Privacy Risks Through Gradients in Machine Learning](https://mlsanity.com/paper/2408.16913/) (2024)
- [System-Aware Unlearning Algorithms: Use Lesser, Forget Faster](https://mlsanity.com/paper/2506.06073/) (2025)
- [Differentially Private Synthetic Data Generation via Lipschitz-Regularised Variational Autoencoders](https://mlsanity.com/paper/2304.11336/) (2023)
- [Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profile](https://mlsanity.com/paper/2306.15790/) (2023)
- [Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping](https://mlsanity.com/paper/2403.02051/) (2024)
- [Learning with User-Level Local Differential Privacy](https://mlsanity.com/paper/2405.17079/) (2024)
- [Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization](https://mlsanity.com/paper/2410.19361/) (2024)
- [Dynamic Network-Assisted D2D-Aided Coded Distributed Learning](https://mlsanity.com/paper/2111.14789/) (2021)
- [Federated Variational Inference: Towards Improved Personalization and Generalization](https://mlsanity.com/paper/2305.13672/) (2023)
- [Enhancing Learning with Label Differential Privacy by Vector Approximation](https://mlsanity.com/paper/2405.15150/) (2024)
- [Generalization in Federated Learning: A Conditional Mutual Information Framework](https://mlsanity.com/paper/2503.04091/) (2025)
- [The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches](https://mlsanity.com/paper/2505.24603/) (2025)
- [Towards Differential Relational Privacy and its use in Question Answering](https://mlsanity.com/paper/2203.16701/) (2022)
- [Data Leakage via Access Patterns of Sparse Features in Deep Learning-based Recommendation Systems](https://mlsanity.com/paper/2212.06264/) (2022)

## Abstract (arXiv)

Differential privacy is a precise mathematical constraint meant to ensure privacy of individual pieces of information in a database even while queries are being answered about the aggregate. Intuitively, one must come to terms with what differential privacy does and does not guarantee. For example, the definition prevents a strong adversary who knows all but one entry in the database from further inferring about the last one. This strong adversary assumption can be overlooked, resulting in misinterpretation of the privacy guarantee of differential privacy. Herein we give an equivalent definition of privacy using mutual information that makes plain some of the subtleties of differential privacy. The mutual-information differential privacy is in fact sandwiched between ε-differential privacy and (ε,δ)-differential privacy in terms of its strength. In contrast to previous works using unconditional mutual information, differential privacy is fundamentally related to conditional mutual information, accompanied by a maximization over the database distribution. The conceptual advantage of using mutual information, aside from yielding a simpler and more intuitive definition of differential privacy, is that its properties are well understood. Several properties of differential privacy are easily verified for the mutual information alternative, such as composition theorems.