mlsanityData to 7 Oct 2026

LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses

arXiv 2406.04755 ↗First posted 7 Jun 2024cs.CR cs.AI cs.HCcited by 15

Not on the map. Why?

Abstract (arXiv)

Writing effective prompts for large language models (LLM) can be unintuitive and burdensome. In response, services that optimize or suggest prompts have emerged. While such services can reduce user effort, they also introduce a risk: the prompt provider can subtly manipulate prompts to produce heavily biased LLM responses. In this work, we show that subtle synonym replacements in prompts can increase the likelihood (by a difference up to 78%) that LLMs mention a target concept (e.g., a brand, political party, nation). We substantiate our observations through a user study, showing that our adversarially perturbed prompts 1) are indistinguishable from unaltered prompts by humans, 2) push LLMs to recommend target concepts more often, and 3) make users more likely to notice target concepts, all without arousing suspicion. The practicality of this attack has the potential to undermine user autonomy. Among other measures, we recommend implementing warnings against using prompts from untrusted parties.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 89 of its 170 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

89 of 170 references.

References · 89

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  12. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
38 more
  1. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  2. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  3. 2021Ethical and social risks of harm from Language Modelscited by 957
  4. 2016Adversarial Machine Learning at Scalecited by 934
  5. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  6. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  7. 2019Adversarial Training for Free!cited by 638
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  10. 2017Countering Adversarial Images using Input Transformationscited by 517
  11. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  12. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  13. 2018Generating Natural Language Adversarial Examplescited by 422
  14. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  15. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  16. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  17. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  18. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  19. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  20. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  21. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  22. 2017Detecting Adversarial Samples from Artifactscited by 244
  23. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  24. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  25. 2019Fair Regression: Quantitative Definitions and Reduction-based Algorithmscited by 168
  26. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  27. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  28. 2024Stealing Part of a Production Language Modelcited by 152
  29. 2020To be Robust or to be Fair: Towards Fairness in Adversarial Trainingcited by 137
  30. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  31. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  32. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 111
  33. 2017A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortionscited by 107
  34. 2021A Comprehensive Study on Face Recognition Biases Beyond Demographicscited by 86
  35. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 81
  36. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  37. 2018Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attackscited by 78
  38. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermarkcited by 78

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 7.

15 papers.

Sort:

By area:

  1. 2025Consistency in Language Models: Current Landscape, Challenges, and Future Directionscited by 14
  2. 2025The threat of analytic flexibility in using large language models to simulate human datacited by 13
  3. 2025Estimating LLM Consistency: A User Baseline vs Surrogate Metricscited by 8
  4. 2025Cascading Adversarial Bias from Injection to Distillation in Language Modelscited by 8
  5. 2025Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimizationcited by 8
  6. 2025MaxShapley: Towards Incentive-compatible Generative Search with Fair Context Attributioncited by 3
  7. 2026Multimodal Generative Engine Optimization: Rank Manipulation for Vision-Language Model Rankerscited by 3
  8. 2025You've Changed: Detecting Modification of Black-Box Large Language Modelscited by 2
  9. 2025User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarioscited by 1
  10. 2026Generative AI Advertising as a Problem of Trustworthy Commercial Interventioncited by 1
  11. 2026Persona Conditioning of Brand Recommendations in Retrieval-Augmented Commercial Chat: A Prominence-Stratified Cross-Provider Auditcited by 1
  12. 2026Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systemscited by 1
3 more
  1. 2026Semantic Content Determines Algorithmic Performancecited by 0
  2. 2026Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Modelscited by 0
  3. 2026Implicit Manipulation for Skill Selection in LLM Agents with Semantic Matchingcited by 0