mlsanityData to 7 Oct 2026

BrainWash: A Poisoning Attack to Forget in Continual Learning

arXiv 2311.11995 ↗First posted 20 Nov 2023cs.LG cs.AI cs.CRcited by 9

Not on the map. Why?

Abstract (arXiv)

Continual learning has gained substantial attention within the deep learning community, offering promising solutions to the challenging problem of sequential learning. Yet, a largely unexplored facet of this paradigm is its susceptibility to adversarial attacks, especially with the aim of inducing forgetting. In this paper, we introduce "BrainWash," a novel data poisoning method tailored to impose forgetting on a continual learner. By adding the BrainWash noise to a variety of baselines, we demonstrate how a trained continual learner can be induced to forget its previously learned tasks catastrophically, even when using these continual learning baselines. An important feature of our approach is that the attacker requires no access to previous tasks' data and is armed merely with the model's current parameters and the data belonging to the most recent task. Our extensive experiments highlight the efficacy of BrainWash, showcasing degradation in performance across various regularization-based continual learning methods.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 51 of its 70 references that are in the map. First posted on arXiv 20 Nov 2023.

Builds on

51 of 70 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2016Matching Networks for One Shot Learningcited by 2,246
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  7. 2016Progressive Neural Networkscited by 1,332
  8. 2017Continual Learning with Deep Generative Replaycited by 1,152
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  10. 2018Experience Replay for Continual Learningcited by 1,074
  11. 2019Machine Unlearningcited by 1,020
  12. 2020Designing Network Design Spacescited by 954
38 more
  1. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  2. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  3. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  4. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  5. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  6. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  7. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  8. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
  9. 2021Gradient Projection Memory for Continual Learningcited by 338
  10. 2019Continual learning with hypernetworkscited by 267
  11. 2020Supermasks in Superpositioncited by 223
  12. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  13. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  14. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  15. 2021Unlearnable Examples: Making Personal Data Unexploitablecited by 188
  16. 2018Truncated Back-propagation for Bilevel Optimizationcited by 187
  17. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  18. 2018Towards Robust Evaluations of Continual Learningcited by 148
  19. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  20. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  21. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 114
  22. 2018Generative replay with feedback connections as a general strategy for continual learningcited by 103
  23. 2023GOAT: GO to Any Thingcited by 89
  24. 2022TRGP: Trust Region Gradient Projection for Continual Learningcited by 79
  25. 2022Variational Model Inversion Attackscited by 75
  26. 2021AFEC: Active Forgetting of Negative Transfer in Continual Learningcited by 73
  27. 2019Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encodercited by 70
  28. 2018Algorithms that Remember: Model Inversion Attacks and Data Protection Lawcited by 52
  29. 2023Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learningcited by 41
  30. 2023Re-thinking Model Inversion Attacks Against Deep Neural Networkscited by 39
  31. 2023TiC-CLIP: Continual Training of CLIP Modelscited by 38
  32. 2019Generative Continual Concept Learningcited by 26
  33. 2022Lifelong Reinforcement Learning with Modulating Maskscited by 20
  34. 2022Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activationscited by 18
  35. 2021Lifelong Learning with Sketched Structural Regularizationcited by 15
  36. 2020Targeted Forgetting and False Memory Formation in Continual Learners through Adversarial Backdoor Attackscited by 14
  37. 2023Sharing Lifelong Reinforcement Learning Knowledge via Modulating Maskscited by 10
  38. 2021DeepPoison: Feature Transfer Based Stealthy Poisoning Attackcited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 3; 2026: 3.

9 papers.

  1. 2024Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspacecited by 14
  2. 2023Adversarial Robust Memory-Based Continual Learnercited by 5
  3. 2024Data Poisoning: An Overlooked Threat to Power Grid Resiliencecited by 1
  4. 2025Impact of Data Poisoning Attacks on Feasibility and Optimality of Neural Power System Optimizerscited by 0
  5. 2025Noise-Tolerant Coreset-Based Class Incremental Continual Learningcited by 0
  6. 2025Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learningcited by 0
  7. 2026Amnesia: A Stealthy Replay Attack on Continual Learning Dreamscited by 0
  8. 2026Theory of Continual Learning Against Data Poisoning Attackscited by 0
  9. 2026Anchoring Bias: A Persistent Fairness Backdoor Attack against MLLMs under Continual Learningcited by 0