mlsanityData to 7 Oct 2026

GhostEncoder: Stealthy Backdoor Attacks with Dynamic Triggers to Pre-trained Encoders in Self-supervised Learning

arXiv 2310.00626 ↗First posted 1 Oct 2023cs.CV cs.CRcited by 6

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Within the realm of computer vision, self-supervised learning (SSL) pertains to training pre-trained image encoders utilizing a substantial quantity of unlabeled images. Pre-trained image encoders can serve as feature extractors, facilitating the construction of downstream classifiers for various tasks. However, the use of SSL has led to an increase in security research related to various backdoor attacks. Currently, the trigger patterns used in backdoor attacks on SSL are mostly visible or static (sample-agnostic), making backdoors less covert and significantly affecting the attack performance. In this work, we propose GhostEncoder, the first dynamic invisible backdoor attack on SSL. Unlike existing backdoor attacks on SSL, which use visible or static trigger patterns, GhostEncoder utilizes image steganography techniques to encode hidden information into benign images and generate backdoor samples. We then fine-tune the pre-trained image encoder on a manipulation dataset to inject the backdoor, enabling downstream classifiers built upon the backdoored encoder to inherit the backdoor behavior for target downstream tasks. We evaluate GhostEncoder on three downstream tasks and results demonstrate that GhostEncoder provides practical stealthiness on images and deceives the victim model with a high attack success rate without compromising its utility. Furthermore, GhostEncoder withstands state-of-the-art defenses, including STRIP, STRIP-Cl, and SSL-Cleanse.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 38 of its 46 references that are in the map. First posted on arXiv 1 Oct 2023.

Builds on

38 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Detecting Backdoors in Pre-trained Encoders23 Mar
  2. 2023SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning16 Mar
  3. 2022PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning13 May
  4. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning1 Aug

Other references · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  11. 2015Spatial Transformer Networkscited by 1,423
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
22 more
  1. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  2. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  3. 2016Colorful Image Colorizationcited by 1,113
  4. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  5. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  6. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  7. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  8. 2018HiDDeN: Hiding Data With Deep Networkscited by 384
  9. 2019Hidden Trigger Backdoor Attackscited by 359
  10. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  11. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  12. 2019StegaStamp: Invisible Hyperlinks in Physical Photographscited by 256
  13. 2017Neural Trojanscited by 151
  14. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  15. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  16. 2021Backdoor Scanning for Deep Neural Networks through K-Arm Optimizationcited by 96
  17. 2018Boosting Self-Supervised Learning via Knowledge Transfercited by 92
  18. 2021Backdoor Attacks on Self-Supervised Learningcited by 89
  19. 2022Frequency-driven Imperceptible Adversarial Attack on Semantic Similaritycited by 76
  20. 2021Poison Ink: Robust and Invisible Backdoor Attackcited by 50
  21. 2020Backdooring and Poisoning Neural Networks with Image-Scaling Attackscited by 28
  22. 2023Defending Against Patch-based Backdoor Attacks on Self-Supervised Learningcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 3; 2026: 1.

6 papers.

  1. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 112
  2. 2025Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Modelscited by 13
  3. 2025Backdooring Self-Supervised Contrastive Learning by Noisy Alignmentcited by 6
  4. 2023SSL-OTA: Unveiling Backdoor Threats in Self-Supervised Learning for Object Detectioncited by 2
  5. 2025Assimilation Matters: Model-level Backdoor Detection in Vision-Language Pretrained Modelscited by 0
  6. 2026Repurposing and Evaluating the (In)Feasibility of Dataset Poisoning enabled Watermarking for Contrastive Learningcited by 0