mlsanityData to 7 Oct 2026

Pre-trained Adversarial Perturbations

arXiv 2210.03372 ↗First posted 7 Oct 2022cs.CV cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Self-supervised pre-training has drawn increasing attention in recent years due to its superior performance on numerous downstream tasks after fine-tuning. However, it is well-known that deep learning models lack the robustness to adversarial examples, which can also invoke security issues to pre-trained models, despite being less explored. In this paper, we delve into the robustness of pre-trained models by introducing Pre-trained Adversarial Perturbations (PAPs), which are universal perturbations crafted for the pre-trained models to maintain the effectiveness when attacking fine-tuned ones without any knowledge of the downstream tasks. To this end, we propose a Low-Level Layer Lifting Attack (L4A) method to generate effective PAPs by lifting the neuron activations of low-level layers of the pre-trained models. Equipped with an enhanced noise augmentation strategy, L4A is effective at generating more transferable PAPs against fine-tuned models. Extensive experiments on typical pre-trained vision models and ten downstream tasks demonstrate that our method improves the attack success rate by a large margin compared with state-of-the-art methods.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 50 of its 60 references that are in the map. First posted on arXiv 7 Oct 2022.

Builds on

50 of 60 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  11. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  12. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
38 more
  1. 2013Intriguing properties of neural networkscited by 5,114
  2. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  3. 2020Supervised Contrastive Learningcited by 2,803
  4. 2013Describing Textures in the Wildcited by 2,110
  5. 2016Adversarial examples in the physical worldcited by 1,762
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  7. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  8. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  9. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  11. 2014How transferable are features in deep neural networks?cited by 1,216
  12. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  13. 2016Universal adversarial perturbationscited by 774
  14. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  15. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  16. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  17. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  18. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  19. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  20. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  21. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  22. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  23. 2018SpotTune: Transfer Learning through Adaptive Fine-tuningcited by 166
  24. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 152
  25. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  26. 2018Data Augmentation by Pairing Samples for Images Classificationcited by 96
  27. 2019Cross-Domain Transferability of Adversarial Perturbationscited by 92
  28. 2019ClusterFit: Improving Generalization of Visual Representationscited by 82
  29. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  30. 2017Fast Feature Fool: A data independent approach to universal adversarial perturbationscited by 65
  31. 2020Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretrainingcited by 59
  32. 2017Art of singular vectors and universal adversarial perturbationscited by 49
  33. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  34. 2019Enhancing Cross-task Black-Box Transferability of Adversarial Examples with Dispersion Reductioncited by 47
  35. 2019Selfie: Self-supervised Pretraining for Image Embeddingcited by 46
  36. 2022Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domainscited by 42
  37. 2021Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuningcited by 24
  38. 2022On Robust Prefix-Tuning for Text Classificationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 5; 2025: 3; 2026: 2.

16 papers.

Sort:

By area:

  1. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 119
  2. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  3. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentationcited by 39
  4. 2023Pre-trained Trojan Attacks for Visual Recognitioncited by 38
  5. 2023Downstream-agnostic Adversarial Examplescited by 32
  6. 2023Language-Driven Anchors for Zero-Shot Adversarial Robustnesscited by 23
  7. 2024Securely Fine-tuning Pre-trained Encoders Against Adversarial Examplescited by 21
  8. 2023Adversarial Attacks on Foundational Vision Modelscited by 13
  9. 2024Beyond DAGs: A Latent Partial Causal Model for Multimodal Learningcited by 9
  10. 2025DarkHash: A Data-Free Backdoor Attack Against Deep Hashingcited by 6
  11. 2025Robustness Tokens: Towards Adversarial Robustness of Transformerscited by 2
  12. 2024Sparse and Transferable Universal Singular Vectors Attackcited by 1
4 more
  1. 2024Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformerscited by 1
  2. 2025Lorica: A Synergistic Fine-Tuning Framework for Advancing Personalized Adversarial Robustnesscited by 0
  3. 2026Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoderscited by 0
  4. 2026What Makes Adversarial Examples Transfer Across Deepfake Detectors?cited by 0