mlsanityData to 7 Oct 2026

Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content

arXiv 2402.13926 ↗First posted 21 Feb 2024cs.CL cs.AIcited by 8

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The risks derived from large language models (LLMs) generating deceptive and damaging content have been the subject of considerable research, but even safe generations can lead to problematic downstream impacts. In our study, we shift the focus to how even safe text coming from LLMs can be easily turned into potentially dangerous content through Bait-and-Switch attacks. In such attacks, the user first prompts LLMs with safe questions and then employs a simple find-and-replace post-hoc technique to manipulate the outputs into harmful narratives. The alarming efficacy of this approach in generating toxic content highlights a significant challenge in developing reliable safety guardrails for LLMs. In particular, we stress that focusing on the safety of the verbatim LLM outputs is insufficient and that we also need to consider post-hoc transformations.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 17 of its 21 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

17 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 16

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2021Ethical and social risks of harm from Language Modelscited by 957
  6. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  7. 2023A Watermark for Large Language Modelscited by 667
  8. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  9. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  10. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  11. 2021A Survey on Automated Fact-Checkingcited by 376
  12. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
4 more
  1. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  2. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  3. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 94
  4. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 81

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  2. 2024Bileve: Securing Text Provenance in Large Language Models Against Spoofing with Bi-level Signaturecited by 21
  3. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Modelscited by 19
  4. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  5. 2024Decoding Hate: Exploring Language Models' Reactions to Hate Speechcited by 6
  6. 2026TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agentscited by 4
  7. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  8. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1