# Continual Learning with Differential Privacy

arXiv: https://arxiv.org/abs/2110.05223 · HTML: https://mlsanity.com/paper/2110.05223/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Task-Free Continual Learning](https://mlsanity.com/paper/1812.03596/) (2018)
- [Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning](https://mlsanity.com/paper/1904.03137/) (2019)
- [Conditional Channel Gated Networks for Task-Aware Continual Learning](https://mlsanity.com/paper/2004.00070/) (2020)
- [iTAML: An Incremental Task-Agnostic Meta-learning Approach](https://mlsanity.com/paper/2003.11652/) (2020)
- [Memory Replay GANs: learning to generate images from new categories without forgetting](https://mlsanity.com/paper/1809.02058/) (2018)
- [Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning](https://mlsanity.com/paper/1709.05750/) (2017)
- [The Value of Collaboration in Convex Machine Learning with Differential Privacy](https://mlsanity.com/paper/1906.09679/) (2019)
- [Deep learning for neuroimaging: a validation study](https://mlsanity.com/paper/1312.5847/) (2013)
- [Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustness](https://mlsanity.com/paper/1906.01444/) (2019)
- [Differentially Private Continual Learning](https://mlsanity.com/paper/1902.06497/) (2019)
- [Scalable Differential Privacy with Certified Robustness in Adversarial Learning](https://mlsanity.com/paper/1903.09822/) (2019)

## Built on by (most cited first, 6 of 6)

- [Continual Learning with Foundation Models: An Empirical Study of Latent Replay](https://mlsanity.com/paper/2205.00329/) (2022)
- [Federated Continual Learning: Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2502.07059/) (2025)
- [Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning](https://mlsanity.com/paper/2207.12831/) (2022)
- [Forget What's Sensitive, Remember What Matters: Token-Level Differential Privacy in Memory Sculpting for Continual Learning](https://mlsanity.com/paper/2509.12958/) (2025)
- [What is the role of memorization in Continual Learning?](https://mlsanity.com/paper/2505.17664/) (2025)
- [Continual Learning for fMRI-Based Brain Disorder Diagnosis via Functional Connectivity Matrices Generative Replay](https://mlsanity.com/paper/2604.14259/) (2026)

## Abstract (arXiv)

In this paper, we focus on preserving differential privacy (DP) in continual learning (CL), in which we train ML models to learn a sequence of new tasks while memorizing previous tasks. We first introduce a notion of continual adjacent databases to bound the sensitivity of any data record participating in the training process of CL. Based upon that, we develop a new DP-preserving algorithm for CL with a data sampling strategy to quantify the privacy risk of training data in the well-known Averaged Gradient Episodic Memory (A-GEM) approach by applying a moments accountant. Our algorithm provides formal guarantees of privacy for data records across tasks in CL. Preliminary theoretical analysis and evaluations show that our mechanism tightens the privacy loss while maintaining a promising model utility.