mlsanityData to 7 Oct 2026

Downstream-agnostic Adversarial Examples

arXiv 2307.12280 ↗First posted 23 Jul 2023cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning usually uses a large amount of unlabeled data to pre-train an encoder which can be used as a general-purpose feature extractor, such that downstream users only need to perform fine-tuning operations to enjoy the benefit of "large model". Despite this promising prospect, the security of pre-trained encoder has not been thoroughly investigated yet, especially when the pre-trained encoder is publicly available for commercial use. In this paper, we propose AdvEncoder, the first framework for generating downstream-agnostic universal adversarial examples based on the pre-trained encoder. AdvEncoder aims to construct a universal adversarial perturbation or patch for a set of natural images that can fool all the downstream tasks inheriting the victim pre-trained encoder. Unlike traditional adversarial example works, the pre-trained encoder only outputs feature vectors rather than classification labels. Therefore, we first exploit the high frequency component information of the image to guide the generation of adversarial examples. Then we design a generative attack framework to construct adversarial perturbations/patches by learning the distribution of the attack surrogate dataset to improve their attack success rates and transferability. Our results show that an attacker can successfully attack downstream tasks without knowing either the pre-training dataset or the downstream dataset. We also tailor four defenses for pre-trained encoders, the results of which further prove the attack ability of AdvEncoder.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 23 Jul 2023.

Builds on

53 of 71 references.

References · 53

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2020Supervised Contrastive Learningcited by 2,803
38 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  3. 2016Adversarial examples in the physical worldcited by 1,762
  4. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  6. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  7. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  8. 2016Universal adversarial perturbationscited by 774
  9. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  10. 2017Adversarial Patchcited by 514
  11. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  12. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  13. 2020Whitening for Self-Supervised Representation Learningcited by 278
  14. 2019Adversarial Training and Robustness for Multiple Perturbationscited by 182
  15. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  16. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  17. 2021Self-Supervised Models are Continual Learnerscited by 149
  18. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  19. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learningcited by 120
  20. 2021Solo-learn: A Library of Self-supervised Methods for Visual Representation Learningcited by 116
  21. 2021Backdoor Attacks on Self-Supervised Learningcited by 89
  22. 2018Generalizable Data-free Objective for Crafting Universal Adversarial Perturbationscited by 88
  23. 2017Measuring the tendency of CNNs to Learn Surface Statistical Regularitiescited by 87
  24. 2021ReSSL: Relational Self-Supervised Learning with Weak Augmentationcited by 83
  25. 2022Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfercited by 77
  26. 2021EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learningcited by 68
  27. 2017Fast Feature Fool: A data independent approach to universal adversarial perturbationscited by 65
  28. 2022Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbationscited by 58
  29. 2017NAG: Network for Adversary Generationcited by 51
  30. 2021Exploring the Equivalence of Siamese Self-Supervised Learning via A Unified Gradient Frameworkcited by 49
  31. 2022SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoderscited by 43
  32. 2021Quantifying and Mitigating Privacy Risks of Contrastive Learningcited by 35
  33. 2022PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learningcited by 33
  34. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  35. 2018Ask, Acquire, and Attack: Data-free UAP Generation using Class Impressionscited by 31
  36. 2022Can't Steal? Cont-Steal! Contrastive Stealing Attacks Against Image Encoderscited by 30
  37. 2022BadHash: Invisible Backdoor Attacks against Deep Hashing with Clean Labelcited by 28
  38. 2019Design and Interpretation of Universal Adversarial Patches in Face Detectioncited by 22

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 11; 2025: 10; 2026: 7.

32 papers.

Sort:

By area:

  1. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  2. 2024One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Modelscited by 39
  3. 2024DarkSAM: Fooling Segment Anything Model to Segment Nothingcited by 32
  4. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
  5. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  6. 2024Securely Fine-tuning Pre-trained Encoders Against Adversarial Examplescited by 21
  7. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  8. 2024Transferable Adversarial Attacks on SAM and Its Downstream Modelscited by 20
  9. 2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agentscited by 19
  10. 2024PB-UAP: Hybrid Universal Adversarial Attack For Image Segmentationcited by 14
  11. 2025Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2cited by 11
  12. 2024ECLIPSE: Expunging Clean-label Indiscriminate Poisons via Sparse Diffusion Purificationcited by 9
20 more
  1. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verificationcited by 8
  2. 2025SegTrans: Transferable Adversarial Examples for Segmentation Modelscited by 8
  3. 2023Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systemscited by 6
  4. 2025SITA: Structurally Imperceptible and Transferable Adversarial Attacks for Stylized Image Generationcited by 6
  5. 2025DarkHash: A Data-Free Backdoor Attack Against Deep Hashingcited by 6
  6. 2025Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protectioncited by 5
  7. 2025Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Modelscited by 5
  8. 2023SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbationcited by 4
  9. 2026Erosion Attack for Adversarial Training to Enhance Semantic Segmentation Robustnesscited by 3
  10. 2024New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlookcited by 2
  11. 2026Transferable Physical-World Adversarial Patches Against Pedestrian Detection Modelscited by 2
  12. 2024Sparse and Transferable Universal Singular Vectors Attackcited by 1
  13. 2024Detector Collapse: Physical-World Backdooring Object Detection to Catastrophic Overload or Blindness in Autonomous Drivingcited by 1
  14. 2024Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformerscited by 1
  15. 2025Towards Real-World Deepfake Detection: A Diverse In-the-wild Dataset of Forgery Facescited by 1
  16. 2026Transferable Physical-World Adversarial Patches Against Object Detection in Autonomous Drivingcited by 1
  17. 2026Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoderscited by 0
  18. 2026The Unseen Adversaries: Robust and Generalized Defense Against Adversarial Patchescited by 0
  19. 2026VFACamou: View-Fused Adversarial Camouflage for Environment-Adaptive Physical Evasioncited by 0
  20. 2026Universal Cross-Prompt Adversarial Attacks on Promptable Concept Segmentationcited by 0