Efficient LLM-Jailbreaking via Multimodal-LLM Jailbreak
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
This paper focuses on jailbreaking attacks against large language models (LLMs), eliciting them to generate objectionable content in response to harmful user queries. Unlike previous LLM-jailbreak methods that directly orient to LLMs, our approach begins by constructing a multimodal large language model (MLLM) built upon the target LLM. Subsequently, we perform an efficient MLLM jailbreak and obtain a jailbreaking embedding. Finally, we convert the embedding into a textual jailbreaking suffix to carry out the jailbreak of target LLM. Compared to the direct LLM-jailbreak methods, our indirect jailbreaking approach is more efficient, as MLLMs are more vulnerable to jailbreak than pure LLM. Additionally, to improve the attack success rate of jailbreak, we propose an image-text semantic matching scheme to identify a suitable initial input. Extensive experiments demonstrate that our approach surpasses current state-of-the-art jailbreak methods in terms of both efficiency and effectiveness. Moreover, our approach exhibits superior cross-class generalization abilities.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 17 of its 31 references that are in the map. First posted on arXiv 30 May 2024.
Builds on
17 of 31 references.
Closest ancestors · 2
Same direction, newest first
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 15
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
- 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
Built on by
Citing papers per year: 2024: 3; 2025: 1.
4 papers.
- 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
- 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
- 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMscited by 2
- 2025Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attackscited by 1
- 2025Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks16 Oct 2025
- 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMs28 Dec 2024
- 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey14 Nov 2024
- 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerations9 Oct 2024
Similar, not cited
Close in wording, no citation either way
- 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
- 2024Jailbreaking Attack against Multimodal Large Language Model
- 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking
- 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
- 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
- 2025Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application
- 2026DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs
- 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models