# Why do universal adversarial attacks work on large language models?: Geometry might be the answer

arXiv: https://arxiv.org/abs/2309.00254 · HTML: https://mlsanity.com/paper/2309.00254/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 1 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks](https://mlsanity.com/paper/1608.04207/) (2016)
- [Visualizing and Measuring the Geometry of BERT](https://mlsanity.com/paper/1906.02715/) (2019)
- [What do Neural Machine Translation Models Learn about Morphology?](https://mlsanity.com/paper/1704.03471/) (2017)
- [Open Sesame: Getting Inside BERT's Linguistic Knowledge](https://mlsanity.com/paper/1906.01698/) (2019)
- [Universal Adversarial Attacks with Natural Triggers for Text Classification](https://mlsanity.com/paper/2005.00174/) (2020)
- [Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models](https://mlsanity.com/paper/2009.07053/) (2020)
- [BERTnesia: Investigating the capture and forgetting of knowledge in BERT](https://mlsanity.com/paper/2106.02902/) (2021)
- ["That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks](https://mlsanity.com/paper/2204.04636/) (2022)
- [HotFlip: White-Box Adversarial Examples for Text Classification](https://mlsanity.com/paper/1712.06751/) (2017)
- [Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey](https://mlsanity.com/paper/1901.06796/) (2019)
- [MINIMAL: Mining Models for Data Free Universal Adversarial Triggers](https://mlsanity.com/paper/2109.12406/) (2021)
- [Robustness of Explanation Methods for NLP Models](https://mlsanity.com/paper/2206.12284/) (2022)

## Built on by (most cited first, 14 of 14)

- [AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks](https://mlsanity.com/paper/2403.04783/) (2024)
- [Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models](https://mlsanity.com/paper/2403.09792/) (2024)
- [A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models](https://mlsanity.com/paper/2402.13457/) (2024)
- [Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective](https://mlsanity.com/paper/2401.06824/) (2024)
- [RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process](https://mlsanity.com/paper/2410.08660/) (2024)
- [Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling](https://mlsanity.com/paper/2510.01025/) (2025)
- [Asymmetric Bias in Text-to-Image Generation with Adversarial Attacks](https://mlsanity.com/paper/2312.14440/) (2023)
- [Can Large Language Models Automatically Jailbreak GPT-4V?](https://mlsanity.com/paper/2407.16686/) (2024)
- [ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Concepts](https://mlsanity.com/paper/2508.16325/) (2025)
- [Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference](https://mlsanity.com/paper/2408.01935/) (2024)
- [Manifold of Failure: Behavioral Attraction Basins in Language Models](https://mlsanity.com/paper/2602.22291/) (2026)
- [Structured Universal Adversarial Attacks on Object Detection for Video Sequences](https://mlsanity.com/paper/2510.14460/) (2025)
- [Breaking the Adversarial Robustness-Performance Trade-off in Text Classification via Manifold Purification](https://mlsanity.com/paper/2511.07888/) (2025)
- [Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study](https://mlsanity.com/paper/2608.27504/) (2026)

## Abstract (arXiv)

Transformer based large language models with emergent capabilities are becoming increasingly ubiquitous in society. However, the task of understanding and interpreting their internal workings, in the context of adversarial attacks, remains largely unsolved. Gradient-based universal adversarial attacks have been shown to be highly effective on large language models and potentially dangerous due to their input-agnostic nature. This work presents a novel geometric perspective explaining universal adversarial attacks on large language models. By attacking the 117M parameter GPT-2 model, we find evidence indicating that universal adversarial triggers could be embedding vectors which merely approximate the semantic information in their adversarial training region. This hypothesis is supported by white-box model analysis comprising dimensionality reduction and similarity measurement of hidden representations. We believe this new geometric perspective on the underlying mechanism driving universal attacks could help us gain deeper insight into the internal workings and failure modes of LLMs, thus enabling their mitigation.