mlsanityData to 7 Oct 2026

PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning

arXiv 2205.06401 ↗First posted 13 May 2022cs.CR cs.CV cs.LGcited by 33

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Contrastive learning pre-trains an image encoder using a large amount of unlabeled data such that the image encoder can be used as a general-purpose feature extractor for various downstream tasks. In this work, we propose PoisonedEncoder, a data poisoning attack to contrastive learning. In particular, an attacker injects carefully crafted poisoning inputs into the unlabeled pre-training data, such that the downstream classifiers built based on the poisoned encoder for multiple target downstream tasks simultaneously classify attacker-chosen, arbitrary clean inputs as attacker-chosen, arbitrary classes. We formulate our data poisoning attack as a bilevel optimization problem, whose solution is the set of poisoning inputs; and we propose a contrastive-learning-tailored method to approximately solve it. Our evaluation on multiple datasets shows that PoisonedEncoder achieves high attack success rates while maintaining the testing accuracy of the downstream classifiers built upon the poisoned encoder for non-attacker-chosen inputs. We also evaluate five defenses against PoisonedEncoder, including one pre-processing, three in-processing, and one post-processing defenses. Our results show that these defenses can decrease the attack success rate of PoisonedEncoder, but they also sacrifice the utility of the encoder or require a large clean pre-training dataset.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 43 of its 60 references that are in the map. First posted on arXiv 13 May 2022.

Builds on

43 of 60 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning1 Aug
  2. 2021De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks8 May
  3. 2021Data Poisoning Attacks and Defenses to Crowdsourcing Systems18 Feb

Other references · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  7. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  8. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  9. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  10. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  11. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  12. 2018Spectral Signatures in Backdoor Attackscited by 457
28 more
  1. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  2. 2017Certified Defenses for Data Poisoning Attackscited by 288
  3. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  4. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 156
  5. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  6. 2018Stronger Data Poisoning Attacks Break Data Sanitization Defensescited by 125
  7. 2020You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completioncited by 112
  8. 2020Certified Robustness to Label-Flipping Attacks via Randomized Smoothingcited by 103
  9. 2020Intrinsic Certified Robustness of Bagging against Data Poisoning Attackscited by 90
  10. 2016Data Poisoning Attacks on Factorization-Based Collaborative Filteringcited by 78
  11. 2018Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attackscited by 78
  12. 2019Data Poisoning against Differentially-Private Learners: Attacks and Defensescited by 75
  13. 2020On Certifying Robustness against Backdoor Attacks via Randomized Smoothingcited by 74
  14. 2018Label Sanitization against Label Flipping Poisoning Attackscited by 68
  15. 2020Subpopulation Data Poisoning Attackscited by 68
  16. 2022MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clientscited by 65
  17. 2020Influence Function based Data Poisoning Attacks to Top-N Recommender Systemscited by 61
  18. 2020Certified Robustness of Nearest Neighbors against Data Poisoning and Backdoor Attackscited by 58
  19. 2020Certified Robustness of Graph Neural Networks against Adversarial Structural Perturbationcited by 57
  20. 2019Attacking Graph-based Classification via Manipulating the Graph Structurecited by 53
  21. 2018Poisoning Attacks to Graph-Based Recommender Systemscited by 52
  22. 2021Poisoning the Unlabeled Dataset of Semi-Supervised Learningcited by 44
  23. 2018Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detectioncited by 43
  24. 2021Data Poisoning Attacks to Deep Learning Based Recommender Systemscited by 41
  25. 2020Certified Robustness of Community Detection against Adversarial Structural Perturbation via Randomized Smoothingcited by 29
  26. 2019Data Poisoning Attacks to Local Differential Privacy Protocolscited by 20
  27. 2020Humpty Dumpty: Controlling Word Meanings via Corpus Poisoningcited by 17
  28. 2021Poisoning Attacks to Local Differential Privacy Protocols for Key-Value Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 9; 2025: 4; 2026: 7.

33 papers.

Sort:

By area:

  1. 2023Poisoning Language Models During Instruction Tuningcited by 254
  2. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  3. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  4. 2023Downstream-agnostic Adversarial Examplescited by 32
  5. 2022An Embarrassingly Simple Backdoor Attack on Self-supervised Learningcited by 30
  6. 2022CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learningcited by 28
  7. 2024Securely Fine-tuning Pre-trained Encoders Against Adversarial Examplescited by 21
  8. 2023SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learningcited by 11
  9. 2022ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Samplecited by 9
  10. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  11. 2024A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capabilitycited by 9
  12. 2023On the Difficulty of Defending Contrastive Learning against Backdoor Attackscited by 8
21 more
  1. 2023GhostEncoder: Stealthy Backdoor Attacks with Dynamic Triggers to Pre-trained Encoders in Self-supervised Learningcited by 6
  2. 2023Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systemscited by 6
  3. 2024DeDe: Detecting Backdoor Samples for SSL Encoders via Decoderscited by 6
  4. 2025Backdooring Self-Supervised Contrastive Learning by Noisy Alignmentcited by 6
  5. 2024Invisible Backdoor Attack against Self-supervised Learningcited by 5
  6. 2024Mudjacking: Patching Backdoor Vulnerabilities in Foundation Modelscited by 4
  7. 2023REaaS: Enabling Adversarially Robust Downstream Classifiers via Robust Encoder as a Servicecited by 3
  8. 2023INK: Inheritable Natural Backdoor Attack Against Model Distillationcited by 2
  9. 2024Exploiting Supervised Poison Vulnerability to Strengthen Self-Supervised Defensecited by 1
  10. 2025A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capabilitycited by 1
  11. 2026BackdoorIDS: Zero-shot Backdoor Detection for Pretrained Vision Encodercited by 1
  12. 2024Towards Adversarial Robustness And Backdoor Mitigation in SSLcited by 0
  13. 2024Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Servicescited by 0
  14. 2025TrojanDec: Data-free Detection of Trojan Inputs in Self-supervised Learningcited by 0
  15. 2025Lorica: A Synergistic Fine-Tuning Framework for Advancing Personalized Adversarial Robustnesscited by 0
  16. 2026Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoderscited by 0
  17. 2026Robustness of Vision Foundation Models to Common Perturbationscited by 0
  18. 2026Repurposing and Evaluating the (In)Feasibility of Dataset Poisoning enabled Watermarking for Contrastive Learningcited by 0
  19. 2026The Platonic Defense: Backdoor Defense for Self-Supervised Encoders in the Era of Large Scale Pre-trainingcited by 0
  20. 2026DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priorscited by 0
  21. 2026TraceGuard: Adaptive Multimodal Poison Filtering through Cross-Feature Rank Agreementcited by 0