mlsanityData to 14 Oct 2026

Text-Anchored Semantic Perturbations for Transferable Jailbreak Attacks on Multimodal Large Language Models

arXiv 2608.22312 ↗First posted 23 Aug 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language interaction, yet their safety alignment remains vulnerable to jailbreak attacks. A key challenge is that safety behavior learned in the textual space does not reliably transfer to fused cross-modal representations, leaving multimodal inputs exploitable through latent semantic cues. We propose Text-Anchored Semantic Perturbation Attack (TA-SPA), a black-box jailbreak framework that optimizes transferable perturbations in a text-anchored semantic space. TA-SPA integrates Text-Anchored Semantic Factorization (TASF), which encourages the separation of cross-modal semantic factors from modality-specific residuals, with Semantic-Preserving Augmentation (SPA), which diversifies harmful target anchors while preserving semantic consistency. Experiments show strong attack effectiveness and transfer to commercial MLLMs, with competitive performance under representative defenses. Additional controls and probing support the intended factorization without implying perfect disentanglement, motivating representation-level safety alignment beyond input-level filtering.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 54 references that are in the map. First posted on arXiv 23 Aug 2026.

Builds on

27 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges8 Jun
  2. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves29 Oct
  3. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks28 Oct
  4. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting14 Mar
  5. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  9. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  10. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  11. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 265
  12. 2019Enhancing Adversarial Example Transferability with an Intermediate Level Attackcited by 140
10 more
  1. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 116
  2. 2017Deceiving Google's Perspective API Built for Detecting Toxic Commentscited by 102
  3. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 102
  4. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 62
  5. 2025OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preferencecited by 28
  6. 2023ELF: An End-to-end Local and Global Multimodal Fusion Framework for Glaucoma Gradingcited by 3
  7. 2025Evaluating Prompting Strategies and Large Language Models in Systematic Literature Review Screening: Relevance and Task-Stage Classificationcited by 3
  8. 2024Towards Robust Transcription: Exploring Noise Injection Strategies for Training Data Augmentationcited by 2
  9. 2022Sanitization of Multimedia Content: A Survey of Techniques, Attacks, and Future Directionscited by 1
  10. 2026Efficient Adversarial Training via Criticality-Aware Fine-Tuningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benign Inputs, Harmful Outputs: Cross-Modal Jailbreaking via Distributed Semantic Recomposition
  2. 2025PolyJailbreak: Cross-Modal Jailbreaking Attacks on Black-Box Multimodal LLMs
  3. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
  4. 2025FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction
  5. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attack
  6. 2025DefenSee: Dissecting Threat from Sight and Text -- A Multi-View Defensive Pipeline for Multi-modal Jailbreaks
  7. 2024UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models
  8. 2025TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models