mlsanityData to 7 Oct 2026

Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluation

arXiv 2507.11968 ↗First posted 16 Jul 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) are increasingly used for content moderation, yet their robustness in short-form video contexts remains underexplored. Current safety evaluations often rely on unimodal attacks, failing to address combined attack vulnerabilities. In this paper, we introduce a comprehensive framework for evaluating the tri-modal safety of MLLMs. First, we present the Short-Video Multimodal Adversarial (SVMA) dataset, comprising diverse short-form videos with human-guided synthetic adversarial attacks. Second, we propose ChimeraBreak, a novel tri-modal attack strategy that simultaneously challenges visual, auditory, and semantic reasoning pathways. Extensive experiments on state-of-the-art MLLMs reveal significant vulnerabilities with high Attack Success Rates (ASR). Our findings uncover distinct failure modes, showing model biases toward misclassifying benign or policy-violating content. We assess results using LLM-as-a-judge, demonstrating attack reasoning efficacy. Our dataset and findings provide crucial insights for developing more robust and safe MLLMs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 43 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

34 of 43 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreaking15 Feb
  2. 2024Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models27 Nov
  3. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models31 Oct
  4. 2024Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts21 Jul
  5. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  6. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  7. 2024Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks1 Feb
  8. 2024Red Teaming Visual Language Models23 Jan
  9. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov

Other references · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2024Mixtral of Expertscited by 1,785
  8. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  9. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  10. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  12. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
13 more
  1. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  2. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  3. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
  4. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 99
  5. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 83
  6. 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
  7. 2025Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategycited by 37
  8. 2024Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Imagecited by 26
  9. 2022Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignmentcited by 17
  10. 2024FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMscited by 14
  11. 2025Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analysescited by 13
  12. 2025Cross-modal Causal Relation Alignment for Video Question Groundingcited by 12
  13. 2025Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreakingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025FinCap: Topic-Aligned Captions for Short-Form Financial YouTube Videoscited by 2
  2. 2026SoundBreak: A Systematic Study of Audio-Only Adversarial Attacks on Trimodal Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Transferability of Adversarial Attacks in Video-based MLLMs: A Cross-modal Image-to-Video Approach
  2. 2025CAVALRY-V: A Large-Scale Generator Framework for Adversarial Attacks on Video MLLMs
  3. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  4. 2026Jailbreaking Multimodal Large Language Models using Multi-Clip Video
  5. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models
  6. 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations
  7. 2025Survey of Adversarial Robustness in Multimodal Large Language Models
  8. 2025Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities