mlsanityData to 7 Oct 2026

Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases

arXiv 2602.04739 ↗First posted 4 Feb 2026cs.CL cs.AI cs.HCcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 2 more
Abstract (arXiv)

Multimodal large language models (MLLMs) are increasingly deployed in real-world systems, yet their safety under adversarial prompting remains underexplored. We present a two-phase evaluation of MLLM harmlessness using a fixed benchmark of 726 adversarial prompts authored by 26 professional red teamers. Phase 1 assessed GPT-4o, Claude Sonnet 3.5, Pixtral 12B, and Qwen VL Plus; Phase 2 evaluated their successors (GPT-5, Claude Sonnet 4.5, Pixtral Large, and Qwen Omni) yielding 82,256 human harm ratings. Large, persistent differences emerged across model families: Pixtral models were consistently the most vulnerable, whereas Claude models appeared safest due to high refusal rates. Attack success rates (ASR) showed clear alignment drift: GPT and Claude models exhibited increased ASR across generations, while Pixtral and Qwen showed modest decreases. Modality effects also shifted over time: text-only prompts were more effective in Phase 1, whereas Phase 2 produced model-specific patterns, with GPT-5 and Claude 4.5 showing near-equivalent vulnerability across modalities. These findings demonstrate that MLLM harmlessness is neither uniform nor stable across updates, underscoring the need for longitudinal, multimodal benchmarks to track evolving safety behaviour.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 17 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

10 of 17 references.

References · 10

  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  2. 2021Ethical and social risks of harm from Language Modelscited by 957
  3. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  4. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  5. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  6. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  7. 2023Adversarial Attacks and Defenses in Large Language Models: Old and New Threatscited by 49
  8. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safetycited by 38
  9. 2022A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancementcited by 12
  10. 2025Beyond Perception: Evaluating Abstract Visual Reasoning through Multi-Stage Taskcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments
  2. 2025Red Teaming Multimodal Language Models: Evaluating Harm Across Prompt Modalities and Models
  3. 2026Understanding the Effects of Safety Unalignment on Large Language Models
  4. 2025Survey of Adversarial Robustness in Multimodal Large Language Models
  5. 2026Harnessing Textual Refusal Directions for Multimodal Safety
  6. 2025Universal Adversarial Attack on Aligned Multimodal LLMs
  7. 2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectives
  8. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?