# Data Poisoning Attack Aiming the Vulnerability of Continual Learning

arXiv: https://arxiv.org/abs/2211.15875 · HTML: https://mlsanity.com/paper/2211.15875/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 14 of 14)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder](https://mlsanity.com/paper/1905.09027/) (2019)

## Built on by (most cited first, 6 of 6)

- [Persistent Backdoor Attacks in Continual Learning](https://mlsanity.com/paper/2409.13864/) (2024)
- [BrainWash: A Poisoning Attack to Forget in Continual Learning](https://mlsanity.com/paper/2311.11995/) (2023)
- [Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning](https://mlsanity.com/paper/2507.04106/) (2025)
- [On the Occurence of Critical Learning Periods in Neural Networks](https://mlsanity.com/paper/2510.09687/) (2025)
- [Amnesia: A Stealthy Replay Attack on Continual Learning Dreams](https://mlsanity.com/paper/2606.12655/) (2026)
- [Theory of Continual Learning Against Data Poisoning Attacks](https://mlsanity.com/paper/2606.29841/) (2026)

## Abstract (arXiv)

Generally, regularization-based continual learning models limit access to the previous task data to imitate the real-world constraints related to memory and privacy. However, this introduces a problem in these models by not being able to track the performance on each task. In essence, current continual learning methods are susceptible to attacks on previous tasks. We demonstrate the vulnerability of regularization-based continual learning methods by presenting a simple task-specific data poisoning attack that can be used in the learning process of a new task. Training data generated by the proposed attack causes performance degradation on a specific task targeted by the attacker. We experiment with the attack on the two representative regularization-based continual learning methods, Elastic Weight Consolidation (EWC) and Synaptic Intelligence (SI), trained with variants of MNIST dataset. The experiment results justify the vulnerability proposed in this paper and demonstrate the importance of developing continual learning models that are robust to adversarial attacks.