mlsanityData to 7 Oct 2026

Backpropagation Clipping for Deep Learning with Differential Privacy

arXiv 2202.05089 ↗First posted 10 Feb 2022cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We present backpropagation clipping, a novel variant of differentially private stochastic gradient descent (DP-SGD) for privacy-preserving deep learning. Our approach clips each trainable layer's inputs (during the forward pass) and its upstream gradients (during the backward pass) to ensure bounded global sensitivity for the layer's gradient; this combination replaces the gradient clipping step in existing DP-SGD variants. Our approach is simple to implement in existing deep learning frameworks. The results of our empirical evaluation demonstrate that backpropagation clipping provides higher accuracy at lower values for the privacy parameter ε compared to previous work. We achieve 98.7% accuracy for MNIST with ε = 0.07 and 74% accuracy for CIFAR-10 with ε = 3.64.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 10 Feb 2022.

Builds on

17 of 24 references.

References · 17

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  6. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  7. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  8. 2018Scalable Private Learning with PATEcited by 266
  9. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  10. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 121
  11. 2019Deep Learning with Gaussian Differential Privacycited by 112
  12. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
5 more
  1. 2018The Singular Values of Convolutional Layerscited by 95
  2. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 88
  3. 2020Individual Privacy Accounting via a Renyi Filtercited by 57
  4. 2020Improving Deep Learning with Differential Privacy using Gradient Encoding and Denoisingcited by 30
  5. 2020A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergencescited by 21

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 1.

4 papers.

  1. 2022Debugging Differential Privacy: A Case Study for Privacy Auditingcited by 39
  2. 2023Unleashing the Power of Randomization in Auditing Differentially Private MLcited by 19
  3. 2023Spectral-DP: Differentially Private Deep Learning through Spectral Perturbation and Filteringcited by 6
  4. 2025Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Modelscited by 5