mlsanityData to 7 Oct 2026

FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts

arXiv 2311.05608 ↗First posted 9 Nov 2023cs.CR cs.AI cs.CLcited by 368

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) signify a groundbreaking paradigm shift within the Artificial Intelligence (AI) community, extending beyond the capabilities of Large Language Models (LLMs) by assimilating additional modalities (e.g., images). Despite this advancement, the safety of LVLMs remains adequately underexplored, with a potential overreliance on the safety assurances purported by their underlying LLMs. In this paper, we propose FigStep, a straightforward yet effective black-box jailbreak algorithm against LVLMs. Instead of feeding textual harmful instructions directly, FigStep converts the prohibited content into images through typography to bypass the safety alignment. The experimental results indicate that FigStep can achieve an average attack success rate of 82.50% on six promising open-source LVLMs. Not merely to demonstrate the efficacy of FigStep, we conduct comprehensive ablation studies and analyze the distribution of the semantic embeddings to uncover that the reason behind the success of FigStep is the deficiency of safety alignment for visual embeddings. Moreover, we compare FigStep with five text-only jailbreaks and four image-based jailbreaks to demonstrate the superiority of FigStep, i.e., negligible attack costs and better attack performance. Above all, our work reveals that current LVLMs are vulnerable to jailbreak attacks, which highlights the necessity of novel cross-modality safety alignment techniques. Our code and datasets are available at https://github.com/ThuCCSLab/FigStep .

Key facts. Cited by 368 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 61 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

51 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  2. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2017Deep reinforcement learning from human preferencescited by 3,846
37 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  7. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  12. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  13. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  14. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  15. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  16. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  17. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  18. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  19. 2023Detecting Language Model Attacks with Perplexitycited by 411
  20. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  21. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  22. 2017Deep Reinforcement Learning: An Overviewcited by 327
  23. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  24. 2023Are aligned neural networks adversarially aligned?cited by 305
  25. 2023AI Alignment: A Comprehensive Surveycited by 256
  26. 2023Pretraining Language Models with Human Preferencescited by 245
  27. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  28. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  29. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  30. 2023Model evaluation for extreme riskscited by 167
  31. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  32. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  33. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  34. 2023FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Modelscited by 46
  35. 2020Adversarial Semantic Collisionscited by 42
  36. 2023Exploring Recommendation Capabilities of GPT-4V(ision): A Preliminary Case Studycited by 12
  37. 2023Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 72; 2025: 156; 2026: 138.

368 papers.

Sort:

By area:

  1. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  3. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  4. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  5. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  6. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  7. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  8. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 111
  9. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  10. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  11. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  12. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
38 more
  1. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  2. 2024Best-of-N Jailbreakingcited by 71
  3. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  4. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  5. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  6. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  7. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  8. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
  9. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Modelscited by 57
  10. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  11. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  12. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusalcited by 53
  13. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  14. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  15. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  16. 2024garak: A Framework for Security Probing Large Language Modelscited by 48
  17. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  18. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  19. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Modelscited by 43
  20. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  21. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  22. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaoscited by 39
  23. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  24. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 37
  25. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  26. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  27. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 35
  28. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  29. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 34
  30. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  31. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 33
  32. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  33. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 31
  34. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  35. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreakingcited by 31
  36. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
  37. 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threatscited by 31
  38. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystemscited by 30

50 most cited of 368 All 368 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputs
  2. 2025TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models
  3. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image
  4. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
  5. 2026Jailbreaks on Vision Language Model via Multimodal Reasoning
  6. 2025PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking
  7. 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations
  8. 2025TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models