mlsanityData to 14 Oct 2026

Theory of Continual Learning Against Data Poisoning Attacks

arXiv 2606.29841 ↗First posted 29 Jun 2026cs.LG cs.GTcited by 0

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Continual learning (CL), where a model is trained on a sequence of data tasks, is increasingly being adopted across key fields such as large language models and image recognition, yet it remains highly vulnerable to data poisoning that triggers learning divergence or severe excess risk. Despite these threats, a principled theoretical foundation in CL for understanding attack and defense remains lacking. In this paper, we develop a theoretical framework to analyze strategic attacks and defenses in regularization-based CL, a cornerstone of recent CL theory. By framing the adversary-defender interaction as an online zero-sum game, we first establish a fundamental performance limit: no defense succeeds when an adversary poisons a linear proportion of tasks by injecting unbounded noise or pattern shifts in regularization-based CL. We then analyze two possibly defensible scenarios: infrequent attacks and bounded noise per attack. For the former regime, we propose a task-to-task verification mechanism to detect data poisoning and reduce cumulative bias for learning convergence. For the latter regime, we derive a robust defense that minimizes the model's sensitivity to poisoned features, provably accelerating the convergence rate. Extensive experiments on realistic tasks further validate our theoretical results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 30 of its 37 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

30 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Data Poisoning Attack Aiming the Vulnerability of Continual Learning29 Nov

Other references · 29

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  6. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  7. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  8. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  9. 2018On Lazy Training in Differentiable Programmingcited by 675
  10. 2019Three scenarios for continual learningcited by 614
  11. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 103
  12. 2022How catastrophic can catastrophic forgetting be in linear regression?cited by 81
17 more
  1. 2023Theory on Forgetting and Generalization of Continual Learningcited by 72
  2. 2020Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descentcited by 68
  3. 2020Gradient-based Editing of Memory Examples for Online Task-free Continual Learningcited by 62
  4. 2023Kronecker-Factored Approximate Curvature for Modern Neural Network Architecturescited by 49
  5. 2018Deep Defense: Training DNNs with Improved Adversarial Robustnesscited by 42
  6. 2023The Ideal Continual Learner: An Agent That Never Forgetscited by 35
  7. 2024A Statistical Theory of Regularization-Based Continual Learningcited by 33
  8. 2022Continual Learning with Recursive Gradient Optimizationcited by 32
  9. 2021Continual Learning on Noisy Data Streams via Self-Purified Replaycited by 27
  10. 2022The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shiftcited by 26
  11. 2023Fixed Design Analysis of Regularization-Based Continual Learningcited by 24
  12. 2020Targeted Forgetting and False Memory Formation in Continual Learners through Adversarial Backdoor Attackscited by 15
  13. 2018Attack Strength vs. Detectability Dilemma in Adversarial Machine Learningcited by 14
  14. 2024Persistent Backdoor Attacks in Continual Learningcited by 13
  15. 2023BrainWash: A Poisoning Attack to Forget in Continual Learningcited by 9
  16. 2024LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learningcited by 8
  17. 2019Nonstationary Nonparametric Online Learning: Balancing Dynamic Regret and Model Parsimonycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning
  2. 2023PACOL: Poisoning Attacks Against Continual Learners
  3. 2021Adversarial Targeted Forgetting in Regularization and Generative Based Continual Learning Models
  4. 2026Robust Dynamic Expansion for Continual Learning under Backdoor Attacks via Purification and Selective Recovery
  5. 2022False Memory Formation in Continual Learners Through Imperceptible Backdoor Trigger
  6. 2022Not All Poisons are Created Equal: Robust Training against Data Poisoning
  7. 2025Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning
  8. 2024Flatness-aware Sequential Learning Generates Resilient Backdoors