mlsanityData to 14 Oct 2026

Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025

arXiv 2506.12430 ↗First posted 14 Jun 2025cs.CR cs.CVcited by 12

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have enabled transformative advancements across diverse applications but remain susceptible to safety threats, especially jailbreak attacks that induce harmful outputs. To systematically evaluate and improve their safety, we organized the Adversarial Testing & Large-model Alignment Safety Grand Challenge (ATLAS) 2025}. This technical report presents findings from the competition, which involved 86 teams testing MLLM vulnerabilities via adversarial image-text attacks in two phases: white-box and black-box evaluations. The competition results highlight ongoing challenges in securing MLLMs and provide valuable guidance for developing stronger defense mechanisms. The challenge establishes new benchmarks for MLLM safety evaluation and lays groundwork for advancing safer multimodal AI systems. The code and data for this challenge are openly available at https://github.com/NY1024/ATLAS_Challenge_2025.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 67 of its 74 references that are in the map. First posted on arXiv 14 Jun 2025.

Builds on

67 of 74 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving9 May
  2. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injection19 Apr
  3. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models9 Apr
  4. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreaking15 Feb
  5. 2024RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting25 Dec
  6. 2024Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models27 Nov
  7. 2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Models7 Nov
  8. 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Models21 Jul
  9. 2024Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift27 Jun
  10. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt6 Jun
  11. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security8 Apr
  12. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models21 Feb
4 more
  1. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  2. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  3. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  4. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 51

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  8. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  9. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  10. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 254
  11. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  12. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
38 more
  1. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 115
  2. 2018Transferable Adversarial Attacks for Image and Video Object Detectioncited by 109
  3. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 107
  4. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 96
  5. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  6. 2022Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object Detectioncited by 79
  7. 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 73
  8. 2024On the Role of Attention Heads in Large Language Model Safetycited by 72
  9. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
  10. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 68
  11. 2020Efficient Adversarial Attacks for Visual Object Trackingcited by 66
  12. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 64
  13. 2025Safety in Large Reasoning Models: A Surveycited by 64
  14. 2019Training Robust Deep Neural Networks via Adversarial Noise Propagationcited by 62
  15. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 62
  16. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 59
  17. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 59
  18. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Taskscited by 57
  19. 2022A Large-scale Multiple-objective Method for Black-box Attack against Object Detectioncited by 55
  20. 2021Harnessing Perceptual Adversarial Patches for Crowd Countingcited by 39
  21. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
  22. 2024Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery Detectioncited by 35
  23. 2024GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probingcited by 31
  24. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 27
  25. 2023Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testingcited by 23
  26. 2025Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Modelscited by 22
  27. 2024RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systemscited by 21
  28. 2023RobustMQ: Benchmarking Robustness of Quantized Modelscited by 19
  29. 2023MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Taskscited by 18
  30. 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
  31. 2025Leveraging Multimodal LLM for Inspirational User Interface Searchcited by 13
  32. 2025CogMorph: Cognitive Morphing Attacks for Text-to-Image Modelscited by 12
  33. 2024BDefects4NN: A Backdoor Defect Database for Controlled Localization Studies in Neural Networkscited by 11
  34. 2024DLP: towards active defense against backdoor attacks with decoupled learning processcited by 10
  35. 2025Fairness Mediator: Neutralize Stereotype Associations to Mitigate Bias in Large Language Modelscited by 8
  36. 2024NBA: defensive distillation for backdoor removal via neural behavior alignmentcited by 7
  37. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  38. 2024TrapFlow: Controllable Website Fingerprinting Defense via Dynamic Backdoor Learningcited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
  2. 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 20
  3. 2025SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agentscited by 19
  4. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editingcited by 10
  5. 2025MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Modelscited by 7
  6. 2025SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controllingcited by 3
  7. 2025Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytellingcited by 2
  8. 2026Before Agents Speak: Pre-hoc Failure Risk Inference in Multi-Agent Systemscited by 1
  9. 2025SafeSteer: Adaptive Subspace Steering for Efficient Jailbreak Defense in Vision-Language Modelscited by 0
  10. 2026A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Modelscited by 0
  11. 2026SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defensecited by 0
  12. 2026TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Survey of Adversarial Robustness in Multimodal Large Language Models
  2. 2024Safety of Multimodal Large Language Models on Images and Texts
  3. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  4. 2025Towards Robust Multimodal Large Language Models Against Jailbreak Attacks
  5. 2023How Robust is Google's Bard to Adversarial Image Attacks?
  6. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance
  7. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?
  8. 2026Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases