mlsanityData to 7 Oct 2026

Prompts have evil twins

arXiv 2311.07064 ↗First posted 13 Nov 2023cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

We discover that many natural-language prompts can be replaced by corresponding prompts that are unintelligible to humans but that provably elicit similar behavior in language models. We call these prompts "evil twins" because they are obfuscated and uninterpretable (evil), but at the same time mimic the functionality of the original natural-language prompts (twins). Remarkably, evil twins transfer between models. We find these prompts by solving a maximum-likelihood problem which has applications of independent interest.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

21 of 42 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  10. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
9 more
  1. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  2. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  3. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  4. 2022Can Large Language Models Truly Understand Prompts? A Case Study with Negated Promptscited by 56
  5. 2022Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too?cited by 49
  6. 2022Discovering the Hidden Vocabulary of DALLE-2cited by 42
  7. 2022Adversarial Attacks on Image Generation With Made-Up Wordscited by 29
  8. 2022Robust Conversational Agents against Imperceptible Toxicity Triggerscited by 23
  9. 2023Evaluating the Robustness of Discrete Promptscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMscited by 12
  2. 2024Exploring Prompt Engineering Practices in the Enterprisecited by 11
  3. 2024Evil twins are not that evil: Qualitative insights into machine-generated promptscited by 5
  4. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  5. 2025Inverse Language Modeling towards Robust and Grounded LLMscited by 0
  6. 2026Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibilitycited by 0
  7. 2026BayesPrompt: human readable prompts that make sensecited by 0