mlsanityData to 14 Oct 2026

PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking

arXiv 2507.21540 ↗First posted 29 Jul 2025cs.CR cs.CVcited by 3

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 3 more
Abstract (arXiv)

The increasing sophistication of large vision-language models (LVLMs) has been accompanied by advances in safety alignment mechanisms designed to prevent harmful content generation. However, these defenses remain vulnerable to sophisticated adversarial attacks. Existing jailbreak methods typically rely on direct and semantically explicit prompts, overlooking subtle vulnerabilities in how LVLMs compose information over multiple reasoning steps. In this paper, we propose a novel and effective jailbreak framework inspired by Return-Oriented Programming (ROP) techniques from software security. Our approach decomposes a harmful instruction into a sequence of individually benign visual gadgets. A carefully engineered textual prompt directs the sequence of inputs, prompting the model to integrate the benign visual gadgets through its reasoning process to produce a coherent and harmful output. This makes the malicious intent emergent and difficult to detect from any single component. We validate our method through extensive experiments on established benchmarks including SafeBench and MM-SafetyBench, targeting popular LVLMs. Results show that our approach consistently and substantially outperforms existing baselines on state-of-the-art models, achieving near-perfect attack success rates (over 0.90 on SafeBench) and improving ASR by up to 0.39. Our findings reveal a critical and underexplored vulnerability that exploits the compositional reasoning abilities of LVLMs, highlighting the urgent need for defenses that secure the entire reasoning process.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 18 of its 48 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

18 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models16 Feb

Other references · 17

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  7. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  8. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  9. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 109
  10. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
  11. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 51
  12. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 50
5 more
  1. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
  2. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 33
  3. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 31
  4. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 27
  5. 2024Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Modelscited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editingcited by 10
  2. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 4
  3. 2025Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytellingcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models
  2. 2026Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models
  3. 2026Jailbreaks on Vision Language Model via Multimodal Reasoning
  4. 2025Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense
  5. 2025JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbation
  6. 2025Learning to Detect Unseen Jailbreak Attacks in Large Vision-Language Models
  7. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model
  8. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image