mlsanityData to 7 Oct 2026

How to Sift Out a Clean Data Subset in the Presence of Data Poisoning?

arXiv 2210.06516 ↗First posted 12 Oct 2022cs.CR cs.AI cs.LGcited by 17

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Given the volume of data needed to train modern machine learning models, external suppliers are increasingly used. However, incorporating external data poses data poisoning risks, wherein attackers manipulate their data to degrade model utility or integrity. Most poisoning defenses presume access to a set of clean data (or base set). While this assumption has been taken for granted, given the fast-growing research on stealthy poisoning attacks, a question arises: can defenders really identify a clean subset within a contaminated dataset to support defenses? This paper starts by examining the impact of poisoned samples on defenses when they are mistakenly mixed into the base set. We analyze five defenses and find that their performance deteriorates dramatically with less than 1% poisoned points in the base set. These findings suggest that sifting out a base set with high precision is key to these defenses' performance. Motivated by these observations, we study how precise existing automated tools and human inspection are at identifying clean data in the presence of data poisoning. Unfortunately, neither effort achieves the precision needed. Worse yet, many of the outcomes are worse than random selection. In addition to uncovering the challenge, we propose a practical countermeasure, Meta-Sift. Our method is based on the insight that existing attacks' poisoned samples shifts from clean data distributions. Hence, training on the clean portion of a dataset and testing on the corrupted portion will result in high prediction loss. Leveraging the insight, we formulate a bilevel optimization to identify clean data and further introduce a suite of techniques to improve efficiency and precision. Our evaluation shows that Meta-Sift can sift a clean base set with 100% precision under a wide range of poisoning attacks. The selected base set is large enough to give rise to successful defenses.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 38 of its 66 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

38 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective7 Apr

Other references · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  9. 2016An overview of gradient descent optimization algorithmscited by 1,021
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  11. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  12. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
25 more
  1. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  2. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  3. 2018Spectral Signatures in Backdoor Attackscited by 457
  4. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  5. 2019Hidden Trigger Backdoor Attackscited by 359
  6. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  7. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  8. 2019Label-Consistent Backdoor Attackscited by 285
  9. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  10. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  11. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  12. 2020Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defensescited by 171
  13. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 160
  14. 2020On the Iteration Complexity of Hypergradient Computationcited by 154
  15. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  16. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  17. 2020Adversarial Machine Learning -- Industry Perspectivescited by 121
  18. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  19. 2022Narcissus: A Practical Clean-Label Backdoor Attack with Limited Informationcited by 102
  20. 2020Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Casescited by 93
  21. 2020Rethinking the Trigger of Backdoor Attackcited by 76
  22. 2020DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentationcited by 75
  23. 2021Faster Meta Update Strategy for Noise-Robust Deep Learningcited by 38
  24. 2022Post-Training Detection of Backdoor Attacks for Two-Class and Multi-Attack Scenarioscited by 29
  25. 2019Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statisticcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 6; 2025: 2; 2026: 1.

17 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  3. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  4. 2023A Pathway Towards Responsible AI Generated Contentcited by 38
  5. 2023ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigmscited by 29
  6. 2024Backdoor Secrets Unveiled: Identifying Backdoor Data with Optimized Scaled Prediction Consistencycited by 13
  7. 2023BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detectioncited by 11
  8. 2024CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Modelscited by 11
  9. 2023Temporal Robustness against Data Poisoningcited by 10
  10. 2024Fairness-Aware Meta-Learning via Nash Bargainingcited by 6
  11. 2023Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategycited by 5
  12. 2026Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Modelscited by 2
5 more
  1. 2023On Practical Aspects of Aggregation Defenses against Data Poisoning Attackscited by 1
  2. 2024On the Conflict of Robustness and Learning in Collaborative Machine Learningcited by 1
  3. 2024Mislabeled examples detection viewed as probing machine learning models: concepts, survey and extensive benchmarkcited by 1
  4. 2023Mendata: A Framework to Purify Manipulated Training Datacited by 0
  5. 2025Like Oil and Water: Group Robustness Methods and Poisoning Defenses May Be at Oddscited by 0