mlsanityData to 7 Oct 2026

DeDe: Detecting Backdoor Samples for SSL Encoders via Decoders

arXiv 2411.16154 ↗First posted 25 Nov 2024cs.LG cs.CRcited by 6

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Self-supervised learning (SSL) is pervasively exploited in training high-quality upstream encoders with a large amount of unlabeled data. However, it is found to be susceptible to backdoor attacks merely via polluting a small portion of training data. The victim encoders associate triggered inputs with target embeddings, e.g., mapping a triggered cat image to an airplane embedding, such that the downstream tasks inherit unintended behaviors when the trigger is activated. Emerging backdoor attacks have shown great threats across different SSL paradigms such as contrastive learning and CLIP, yet limited research is devoted to defending against such attacks, and existing defenses fall short in detecting advanced stealthy backdoors. To address the limitations, we propose a novel detection mechanism, DeDe, which detects the activation of backdoor mappings caused by triggered inputs on victim encoders. Specifically, DeDe trains a decoder for any given SSL encoder using an auxiliary dataset (which can be out-of-distribution or even slightly poisoned), so that for any triggered input that misleads the encoder into the target embedding, the decoder generates an output image significantly different from the input. DeDe leverages the discrepancy between the input and the decoded output to identify potential backdoor misbehavior during inference. We empirically evaluate DeDe on both contrastive learning and CLIP models against various types of backdoor attacks. Our results demonstrate promising detection effectiveness over various advanced attacks and superior performance compared over state-of-the-art detection methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 32 of its 48 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

32 of 48 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP26 Nov
  2. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning20 Nov
  3. 2023Detecting Backdoors in Pre-trained Encoders23 Mar
  4. 2023SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning16 Mar
  5. 2023Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks13 Mar
  6. 2023CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning6 Mar
  7. 2022CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learning15 Nov
  8. 2022PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning13 May
  9. 2022Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learning22 Feb
  10. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning1 Aug

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  7. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  8. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  9. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  10. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  11. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  12. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
10 more
  1. 2019Hidden Trigger Backdoor Attackscited by 359
  2. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  3. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  4. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  5. 2020Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Reviewcited by 120
  6. 2021Backdoor Attacks on Self-Supervised Learningcited by 89
  7. 2017Self-supervised learning of visual features through embedding images into text topic spacescited by 43
  8. 2022An Embarrassingly Simple Backdoor Attack on Self-supervised Learningcited by 30
  9. 2023ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigmscited by 29
  10. 2023On the Difficulty of Defending Contrastive Learning against Backdoor Attackscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026When Attention Betrays: Erasing Backdoor Attacks in Robotic Policies by Reconstructing Visual Tokenscited by 4
  2. 2026BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learningcited by 2
  3. 2026BackdoorIDS: Zero-shot Backdoor Detection for Pretrained Vision Encodercited by 1
  4. 2026CLIP-Inspector: Model-Level Backdoor Detection for Prompt-Tuned CLIP via OOD Trigger Inversioncited by 0
  5. 2026The Platonic Defense: Backdoor Defense for Self-Supervised Encoders in the Era of Large Scale Pre-trainingcited by 0
  6. 2026Speculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attackscited by 0