mlsanityData to 7 Oct 2026

Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning

arXiv 2207.12831 ↗First posted 26 Jul 2022cs.LG cs.AI cs.CRcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we show that the process of continually learning new tasks and memorizing previous tasks introduces unknown privacy risks and challenges to bound the privacy loss. Based upon this, we introduce a formal definition of Lifelong DP, in which the participation of any data tuples in the training set of any tasks is protected, under a consistently bounded DP protection, given a growing stream of tasks. A consistently bounded DP means having only one fixed value of the DP privacy budget, regardless of the number of tasks. To preserve Lifelong DP, we propose a scalable and heterogeneous algorithm, called L2DP-ML with a streaming batch training, to efficiently train and continue releasing new versions of an L2M model, given the heterogeneity in terms of data sizes and the training order of tasks, without affecting DP protection of the private training set. An end-to-end theoretical analysis and thorough evaluations show that our mechanism is significantly better than baseline approaches in preserving Lifelong DP. The implementation of L2DP-ML is available at: https://github.com/haiphanNJIT/PrivateDeepLearning.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 26 Jul 2022.

Builds on

30 of 51 references.

References · 30

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  6. 2017Continual Learning with Deep Generative Replaycited by 1,152
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  8. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  9. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  10. 2019Continual learning with hypernetworkscited by 267
  11. 2018Scalable Private Learning with PATEcited by 266
  12. 2020Few-Shot Class-Incremental Learningcited by 249
18 more
  1. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 181
  2. 2020Adversarial Continual Learningcited by 140
  3. 2020Conditional Channel Gated Networks for Task-Aware Continual Learningcited by 115
  4. 2020iTAML: An Incremental Task-Agnostic Meta-learning Approachcited by 90
  5. 2016Towards the Science of Security and Privacy in Machine Learningcited by 89
  6. 2020Learning latent representations across multiple data domains using Lifelong VAEGANcited by 35
  7. 2017Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learningcited by 34
  8. 2020XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learningcited by 30
  9. 2013Deep learning for neuroimaging: a validation studycited by 21
  10. 2021Regularization-based Continual Learning for Anomaly Detection in Discrete Manufacturingcited by 19
  11. 2019Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustnesscited by 15
  12. 2021Online Continual Learning Via Candidates Votingcited by 15
  13. 2019Privacy Accounting and Quality Control in the Sage Differentially Private ML Platformcited by 10
  14. 2017Preserving Differential Privacy in Convolutional Deep Belief Networkscited by 9
  15. 2019Convolutional Neural Networks on Randomized Datacited by 6
  16. 2021Continual Learning with Differential Privacycited by 6
  17. 2017On the Protection of Private Information in Machine Learning Systems: Two Recent Approachescited by 4
  18. 2019Differentially Private Continual Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Privacy-Aware Lifelong Learningcited by 1
  2. 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0