# Adversarial Attacks on Image Generation With Made-Up Words

arXiv: https://arxiv.org/abs/2208.04135 · HTML: https://mlsanity.com/paper/2208.04135/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Bad Characters: Imperceptible NLP Attacks](https://mlsanity.com/paper/2106.09898/) (2021)
- [Discovering the Hidden Vocabulary of DALLE-2](https://mlsanity.com/paper/2206.00169/) (2022)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [Deep Learning and Synthetic Media](https://mlsanity.com/paper/2205.05764/) (2022)

## Built on by (most cited first, 30 of 32)

- [SneakyPrompt: Jailbreaking Text-to-image Generative Models](https://mlsanity.com/paper/2305.12082/) (2023)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Latent Guard: a Safety Framework for Text-to-image Generation](https://mlsanity.com/paper/2404.08031/) (2024)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2407.15861/) (2024)
- [SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution](https://mlsanity.com/paper/2309.14122/) (2023)
- [Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis](https://mlsanity.com/paper/2209.08891/) (2022)
- [Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search](https://mlsanity.com/paper/2306.00974/) (2023)
- [Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks](https://mlsanity.com/paper/2306.13103/) (2023)
- [Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation](https://mlsanity.com/paper/2403.12075/) (2024)
- [Does CLIP Know My Face?](https://mlsanity.com/paper/2209.07341/) (2022)
- [Differentially Private Steering for Large Language Model Alignment](https://mlsanity.com/paper/2501.18532/) (2025)
- [Harnessing LLM to Attack LLM-Guarded Text-to-Image Models](https://mlsanity.com/paper/2312.07130/) (2023)
- [UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers](https://mlsanity.com/paper/2405.11336/) (2024)
- [Revealing Vulnerabilities in Stable Diffusion via Targeted Attacks](https://mlsanity.com/paper/2401.08725/) (2024)
- [A Survey on Responsible Generative AI: What to Generate and What Not](https://mlsanity.com/paper/2404.05783/) (2024)
- [Prompts have evil twins](https://mlsanity.com/paper/2311.07064/) (2023)
- [Autonomous LLM-Enhanced Adversarial Attack for Text-to-Motion](https://mlsanity.com/paper/2408.00352/) (2024)
- [DP-TLDM: Differentially Private Tabular Latent Diffusion Model](https://mlsanity.com/paper/2403.07842/) (2024)
- [Unified Prompt Attack Against Text-to-Image Generation Models](https://mlsanity.com/paper/2502.16423/) (2025)
- [Hypernymy Understanding Evaluation of Text-to-Image Models via WordNet Hierarchy](https://mlsanity.com/paper/2310.09247/) (2023)
- [The Challenges of Image Generation Models in Generating Multi-Component Images](https://mlsanity.com/paper/2311.13620/) (2023)
- [Discovering Universal Semantic Triggers for Text-to-Image Synthesis](https://mlsanity.com/paper/2402.07562/) (2024)
- [Erased, But Not Forgotten: Erased Rectified Flow Transformers Still Remain Unsafe Under Concept Attack](https://mlsanity.com/paper/2510.00635/) (2025)
- [Adversarial Attacks on Parts of Speech: An Empirical Study in Text-to-Image Generation](https://mlsanity.com/paper/2409.15381/) (2024)
- [DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks](https://mlsanity.com/paper/2504.17253/) (2025)
- [Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.06837/) (2025)
- [MacPrompt: Maraconic-guided Jailbreak against Text-to-Image Models](https://mlsanity.com/paper/2601.07141/) (2026)
- [PC2: Politically Controversial Content Generation via Jailbreaking Attacks on GPT-based Text-to-Image Models](https://mlsanity.com/paper/2601.05150/) (2026)
- [Model Hypnosis: Strong control of AI via additive subliminal effects](https://mlsanity.com/paper/2608.16834/) (2026)

## Abstract (arXiv)

Text-guided image generation models can be prompted to generate images using nonce words adversarially designed to robustly evoke specific visual concepts. Two approaches for such generation are introduced: macaronic prompting, which involves designing cryptic hybrid words by concatenating subword units from different languages; and evocative prompting, which involves designing nonce words whose broad morphological features are similar enough to that of existing words to trigger robust visual associations. The two methods can also be combined to generate images associated with more specific visual concepts. The implications of these techniques for the circumvention of existing approaches to content moderation, and particularly the generation of offensive or harmful images, are discussed.