mlsanityData to 14 Oct 2026

Continual Release Moment Estimation with Differential Privacy

arXiv 2502.06597 ↗First posted 10 Feb 2025cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose Joint Moment Estimation (JME), a method for continually and privately estimating both the first and second moments of data with reduced noise compared to naive approaches. JME uses the matrix mechanism and a joint sensitivity analysis to allow the second moment estimation with no additional privacy cost, thereby improving accuracy while maintaining privacy. We demonstrate JME's effectiveness in two applications: estimating the running mean and covariance matrix for Gaussian density estimation, and model training with DP-Adam on CIFAR-10.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

26 of 45 references.

References · 26

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  4. 2013The Composition Theorem for Differential Privacycited by 297
  5. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 218
  6. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 170
  7. 2021Large-Scale Differentially Private BERTcited by 118
  8. 2022Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streamscited by 59
  9. 2023(Amplified) Banded Matrix Factorization: A unified approach to private trainingcited by 49
  10. 2022Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learningcited by 48
  11. 2022New Lower Bounds for Private Estimation and a Generalized Fingerprinting Lemmacited by 33
  12. 2023DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction)cited by 28
14 more
  1. 2022Almost Tight Error Bounds on Differentially Private Continual Countingcited by 26
  2. 2022Constant matters: Fine-grained Complexity of Differentially Private Continual Observationcited by 23
  3. 2023Correlated Noise Provably Beats Independent Noise for Differentially Private Learningcited by 23
  4. 2024Efficient and Near-Optimal Noise Generation for Streaming Differential Privacycited by 17
  5. 2023A Unifying Framework for Differentially Private Sums under Continual Observationcited by 15
  6. 2024Banded Square Root Matrix Factorization for Differentially Private Model Trainingcited by 12
  7. 2022Re-Analyze Gauss: Bounds for Private Matrix Approximation via Dyson Brownian Motioncited by 8
  8. 2024Improved Differentially Private Continual Observation Using Group Algebracited by 7
  9. 2022Exact Privacy Analysis of the Gaussian Sparse Histogram Mechanismcited by 6
  10. 2023Private Covariance Approximation and Eigenvalue-Gap Bounds for Complex Gaussian Perturbationscited by 3
  11. 2024Better Gaussian Mechanism using Correlated Noisecited by 2
  12. 2024The Broader Landscape of Robustness in Algorithmic Statisticscited by 2
  13. 2024The Correlated Gaussian Sparse Histogram Mechanismcited by 2
  14. 2024Privately Counting Partially Ordered Datacited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Back to Square Roots: An Optimal Bound on the Matrix Factorization Error for Multi-Epoch Differentially Private SGDcited by 7
  2. 2025On Design Principles for Private Adaptive Optimizerscited by 6
  3. 2025Binned Group Algebra Factorization for Differentially Private Continual Countingcited by 3
  4. 2025Optimal Variance and Covariance Estimation under Differential Privacy in the Add-Remove Model and Beyondcited by 1
  5. 2026Matrix Factorization for Practical Continual Mean Estimation Under User-Level Differential Privacycited by 0