mlsanityData to 7 Oct 2026

Accumulative Poisoning Attacks on Real-time Data

arXiv 2106.09993 ↗First posted 18 Jun 2021cs.LG cs.CR cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy. When trained on offline datasets, poisoning adversaries have to inject the poisoned data in advance before training, and the order of feeding these poisoned batches into the model is stochastic. In contrast, practical systems are more usually trained/fine-tuned on sequentially captured real-time data, in which case poisoning adversaries could dynamically poison each data batch according to the current model state. In this paper, we focus on the real-time settings and propose a new attacking strategy, which affiliates an accumulative phase with poisoning attacks to secretly (i.e., without affecting accuracy) magnify the destructive effect of a (poisoned) trigger batch. By mimicking online learning and federated learning on MNIST and CIFAR-10, we show that model accuracy significantly drops by a single update step on the trigger batch after the accumulative phase. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects, with no need to explore complex techniques.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 72 of its 100 references that are in the map. First posted on arXiv 18 Jun 2021.

Builds on

72 of 100 references.

References · 72

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  8. 2016Adversarial examples in the physical worldcited by 1,762
  9. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  10. 2018Group Normalizationcited by 1,514
  11. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  12. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
38 more
  1. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  4. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  5. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  6. 2019Hidden Trigger Backdoor Attackscited by 359
  7. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  8. 2017Certified Defenses for Data Poisoning Attackscited by 288
  9. 2019Label-Consistent Backdoor Attackscited by 285
  10. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  11. 2017Detecting Adversarial Samples from Artifactscited by 244
  12. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  13. 2020Bag of Tricks for Adversarial Trainingcited by 199
  14. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  15. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  16. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  17. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  18. 2018Stronger Data Poisoning Attacks Break Data Sanitization Defensescited by 125
  19. 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 123
  20. 2020Adversarial Machine Learning -- Industry Perspectivescited by 121
  21. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 114
  22. 2017Fast Matrix Factorization for Online Recommendation with Implicit Feedbackcited by 100
  23. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  24. 2021Backdoor Attacks on Self-Supervised Learningcited by 89
  25. 2018Is feature selection secure against training data poisoning?cited by 88
  26. 2021Hidden Backdoors in Human-Centric Language Modelscited by 83
  27. 2016Data Poisoning Attacks on Factorization-Based Collaborative Filteringcited by 78
  28. 2018Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attackscited by 78
  29. 2021Manipulating SGD with Data Ordering Attackscited by 73
  30. 2021Black-box Detection of Backdoor Attacks with Limited Information and Datacited by 71
  31. 2020Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferabilitycited by 65
  32. 2018Adversarial Attacks on Stochastic Banditscited by 56
  33. 2019Data Poisoning Attacks on Stochastic Banditscited by 52
  34. 2020Can Adversarial Weight Perturbations Inject Neural Backdoors?cited by 51
  35. 2021A Backdoor Attack against 3D Point Cloud Classifierscited by 50
  36. 2018Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detectioncited by 43
  37. 2021Explainability-based Backdoor Attacks Against Graph Neural Networkscited by 42
  38. 2021PointBA: Towards Backdoor Attacks in 3D Point Cloudcited by 34

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 5; 2024: 1; 2025: 1.

10 papers.

  1. 2025Data Poisoning in Deep Learning: A Surveycited by 20
  2. 2022Can Adversarial Training Be Manipulated By Non-Robust Features?cited by 12
  3. 2023BAFFLE: A Baseline of Backpropagation-Free Federated Learningcited by 12
  4. 2023A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspectivecited by 9
  5. 2022Thinking Two Moves Ahead: Anticipating Other Users Improves Backdoor Attacks in Federated Learningcited by 7
  6. 2022On the Permanence of Backdoors in Evolving Modelscited by 2
  7. 2023PACOL: Poisoning Attacks Against Continual Learnerscited by 1
  8. 2023Attacks on Online Learners: a Teacher-Student Analysiscited by 0
  9. 2023Exploring Model Dynamics for Accumulative Poisoning Discoverycited by 0
  10. 2024Transfer-based Adversarial Poisoning Attacks for Online (MIMO-)Deep Recevierscited by 0