Adversarially Pretrained Transformers May Be Universally Robust In-Context Learners
Not on the map. Why?
Abstract (arXiv)
Adversarial training is one of the most effective defenses against adversarial attacks, but it incurs a high computational cost. In this study, we present the first theoretical analysis suggesting that adversarially pretrained transformers can serve as universally robust foundation models -- models that can adapt robustly to diverse downstream tasks with only lightweight tuning. Specifically, we demonstrate that single-layer linear transformers, after adversarial pretraining across a variety of classification tasks, can generalize robustly to unseen classification tasks through in-context learning from clean demonstrations (i.e., without requiring additional adversarial training or examples). This universal robustness stems from the model's ability to adaptively focus on robust features within given tasks. We also identify two open challenges for attaining robustness: the accuracy-robustness trade-off and sample-hungry training. This study initiates the discussion on the utility of universally robust foundation models. While their training is expensive, the investment would prove worthwhile as downstream tasks can obtain adversarial robustness for free. The code is available at https://github.com/s-kumano/universally-robust-in-context-learner.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 89 of its 106 references that are in the map. First posted on arXiv 20 May 2025.
Builds on
89 of 106 references.
References · 89
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
38 more
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
- 2018Robustness May Be at Odds with Accuracycited by 880
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2022Quantifying Memorization Across Neural Language Modelscited by 763
- 2022Transformers learn in-context by gradient descentcited by 732
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
- 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
- 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
- 2021Intriguing Properties of Vision Transformerscited by 446
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
- 2023Larger language models do in-context learning differentlycited by 394
- 2023Trained Transformers Learn Linear Models In-Contextcited by 337
- 2018Adversarially Robust Generalization Requires More Datacited by 318
- 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
- 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
- 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 279
- 2021Understanding Robustness of Transformers for Image Classificationcited by 249
- 2020Understanding and Improving Fast Adversarial Trainingcited by 214
- 2021Vision Transformers are Robust Learnerscited by 207
- 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
- 2021Are Transformers More Robust Than CNNs?cited by 187
- 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 183
- 2023One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attentioncited by 170
- 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
- 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
- 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
- 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
- 2019Adversarial Training Can Hurt Generalizationcited by 132
- 2023The Learnability of In-Context Learningcited by 127
- 2023What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalizationcited by 118
- 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
50 most cited of 89 All 89 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.