mlsanityData to 7 Oct 2026

Efficient Per-Example Gradient Computations

arXiv 1510.01799 ↗First posted 7 Oct 2015stat.ML cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

This technical report describes an efficient technique for computing the norm of the gradient of the loss function for a neural network with respect to its parameters. This gradient norm can be computed efficiently for every example.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Oct 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 12; 2023: 5; 2024: 6; 2025: 5; 2026: 5.

42 papers.

Sort:

By area:

  1. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  2. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  3. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  4. 2021Deep Learning with Label Differential Privacycited by 113
  5. 2022Toward Training at ImageNet Scale with Differential Privacycited by 87
  6. 2022Differentially Private Optimization on Large Model at Small Costcited by 65
  7. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  8. 2022Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacycited by 47
  9. 2022Differentially Private Bias-Term Fine-tuning of Foundation Modelscited by 47
  10. 2022RelaxLoss: Defending Membership Inference Attacks without Losing Utilitycited by 35
  11. 2021Fast and Memory Efficient Differentially Private-SGD via JL Projectionscited by 29
  12. 2022Introspective Learning : A Two-Stage Approach for Inference in Neural Networkscited by 19
30 more
  1. 2022Model-agnostic out-of-distribution detection using combined statistical testscited by 14
  2. 2022Differentially Private Image Classification from Featurescited by 7
  3. 2023Zero redundancy distributed learning with differential privacycited by 7
  4. 2024Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformerscited by 7
  5. 2021Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliabilitycited by 6
  6. 2022Private Ad Modeling with DP-SGDcited by 6
  7. 2023Private Learning with Public Featurescited by 6
  8. 2024Delving into Differentially Private Transformercited by 5
  9. 2021Learning Deep Neural Networks under Agnostic Corrupted Supervisioncited by 3
  10. 2021Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?cited by 3
  11. 2025R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Trainingcited by 3
  12. 2022Transformer with Memory Replaycited by 2
  13. 2023DPFormer: Learning Differentially Private Transformer on Long-Tailed Datacited by 2
  14. 2024Differentially Private Training of Mixture of Experts Modelscited by 2
  15. 2024DP-MemArc: Differential Privacy Transfer Learning for Memory Efficient Language Modelscited by 2
  16. 2025Learning with Differentially Private (Sliced) Wasserstein Gradientscited by 2
  17. 2025Per-example gradients: a new frontier for understanding and improving optimizerscited by 2
  18. 2021Private Cross-Silo Federated Learning for Extracting Vaccine Adverse Event Mentionscited by 1
  19. 2022Rieoptax: Riemannian Optimization in JAXcited by 1
  20. 2024Noise is All You Need: Private Second-Order Convergence of Noisy SGDcited by 1
  21. 2024Privately Learning from Graphs with Applications in Fine-tuning Large Language Modelscited by 1
  22. 2025FlashDP: Private Training Large Language Models with Efficient DP-SGDcited by 1
  23. 2026Efficient DP-SGD for LLMs with Randomized Clippingcited by 1
  24. 2026Spectral Reach: Understanding Neural Scaling as Progress into the Spectral Tailcited by 1
  25. 2026TAKE: Trajectory-Aware Knowledge Estimation for Text Dataset Distillationcited by 1
  26. 2021Adaptive Optimization with Examplewise Gradientscited by 0
  27. 2023Differentially Private Optimization for Non-Decomposable Objective Functionscited by 0
  28. 2025Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)cited by 0
  29. 2026Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?cited by 0
  30. 2026Every Batch Is Its Own Validation Set: Leave-One-Out Gradient Matching for Online Data Selection in LLM Fine-Tuningcited by 0