# An Adaptive and Fast Convergent Approach to Differentially Private Deep Learning

arXiv: https://arxiv.org/abs/1912.09150 · HTML: https://mlsanity.com/paper/1912.09150/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2019.

## Built on by (most cited first, 13 of 13)

- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [On the Convergence and Calibration of Deep Learning with Differential Privacy](https://mlsanity.com/paper/2106.07830/) (2021)
- [SeqMIA: Sequential-Metric Based Membership Inference Attack](https://mlsanity.com/paper/2407.15098/) (2024)
- [Gradient Leakage Attack Resilient Deep Learning](https://mlsanity.com/paper/2112.13178/) (2021)
- [Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise](https://mlsanity.com/paper/2211.15893/) (2022)
- [Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey](https://mlsanity.com/paper/2309.16398/) (2023)
- [DPSUR: Accelerating Differentially Private Stochastic Gradient Descent Using Selective Update and Release](https://mlsanity.com/paper/2311.14056/) (2023)
- [Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent](https://mlsanity.com/paper/2101.07413/) (2021)
- [Harnessing Inherent Noises for Privacy Preservation in Quantum Machine Learning](https://mlsanity.com/paper/2312.11126/) (2023)
- [SA-DPSGD: Differentially Private Stochastic Gradient Descent based on Simulated Annealing](https://mlsanity.com/paper/2211.07218/) (2022)
- [Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)](https://mlsanity.com/paper/2508.15141/) (2025)
- [Rethinking Layer-wise Gaussian Noise Injection: Bridging Implicit Objectives and Privacy Budget Allocation](https://mlsanity.com/paper/2509.04232/) (2025)
- [Revisiting Privacy Amplification by Subsampling in Selective Release DPSGD](https://mlsanity.com/paper/2606.04384/) (2026)

## Abstract (arXiv)

With the advent of the era of big data, deep learning has become a prevalent building block in a variety of machine learning or data mining tasks, such as signal processing, network modeling and traffic analysis, to name a few. The massive user data crowdsourced plays a crucial role in the success of deep learning models. However, it has been shown that user data may be inferred from trained neural models and thereby exposed to potential adversaries, which raises information security and privacy concerns. To address this issue, recent studies leverage the technique of differential privacy to design private-preserving deep learning algorithms. Albeit successful at privacy protection, differential privacy degrades the performance of neural models. In this paper, we develop ADADP, an adaptive and fast convergent learning algorithm with a provable privacy guarantee. ADADP significantly reduces the privacy cost by improving the convergence speed with an adaptive learning rate and mitigates the negative effect of differential privacy upon the model accuracy by introducing adaptive noise. The performance of ADADP is evaluated on real-world datasets. Experiment results show that it outperforms state-of-the-art differentially private approaches in terms of both privacy cost and model accuracy.