# Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustness

arXiv: https://arxiv.org/abs/1906.01444 · HTML: https://mlsanity.com/paper/1906.01444/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2019.

## Built on by (most cited first, 15 of 15)

- [One Parameter Defense -- Defending against Data Inference Attacks via Differential Privacy](https://mlsanity.com/paper/2203.06580/) (2022)
- [Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learning](https://mlsanity.com/paper/2105.07985/) (2021)
- [Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey](https://mlsanity.com/paper/2309.16398/) (2023)
- [XRand: Differentially Private Defense against Explanation-Guided Attacks](https://mlsanity.com/paper/2212.04454/) (2022)
- [Continual Learning with Differential Privacy](https://mlsanity.com/paper/2110.05223/) (2021)
- [Augment then Smooth: Reconciling Differential Privacy with Certified Robustness](https://mlsanity.com/paper/2306.08656/) (2023)
- [Task-aware Privacy Preservation for Multi-dimensional Data](https://mlsanity.com/paper/2110.02329/) (2021)
- [FairDP: Certified Fairness with Differential Privacy](https://mlsanity.com/paper/2305.16474/) (2023)
- [Wasserstein Differential Privacy](https://mlsanity.com/paper/2401.12436/) (2024)
- [Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning](https://mlsanity.com/paper/2207.12831/) (2022)
- [Combining Machine Learning Defenses without Conflicts](https://mlsanity.com/paper/2411.09776/) (2024)
- [Partial sensitivity analysis in differential privacy](https://mlsanity.com/paper/2109.10582/) (2021)
- [Encoded Gradients Aggregation against Gradient Leakage in Federated Learning](https://mlsanity.com/paper/2205.13216/) (2022)
- [Pure Differential Privacy for Functional Summaries with a Laplace-like Process](https://mlsanity.com/paper/2309.00125/) (2023)
- [An exponential mechanism based on quadratic approximations for fine-tuning machine learning models with privacy guarantees](https://mlsanity.com/paper/2605.20521/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel Heterogeneous Gaussian Mechanism (HGM) to preserve differential privacy in deep neural networks, with provable robustness against adversarial examples. We first relax the constraint of the privacy budget in the traditional Gaussian Mechanism from (0, 1] to (0, ∞), with a new bound of the noise scale to preserve differential privacy. The noise in our mechanism can be arbitrarily redistributed, offering a distinctive ability to address the trade-off between model utility and privacy loss. To derive provable robustness, our HGM is applied to inject Gaussian noise into the first hidden layer. Then, a tighter robustness bound is proposed. Theoretical analysis and thorough evaluations show that our mechanism notably improves the robustness of differentially private deep neural networks, compared with baseline approaches, under a variety of model attacks.