# A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5

arXiv: https://arxiv.org/abs/2601.10527 · HTML: https://mlsanity.com/paper/2601.10527/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 15 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Built on by (most cited first, 8 of 8)

- [Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses](https://mlsanity.com/paper/2605.02900/) (2026)
- [An Independent Safety Evaluation of Kimi K2.5](https://mlsanity.com/paper/2604.03121/) (2026)
- [Constitutional On-Policy Safe Distillation](https://mlsanity.com/paper/2606.03089/) (2026)
- [OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences](https://mlsanity.com/paper/2603.09706/) (2026)
- [Visual Distraction Undermines Moral Reasoning in Vision-Language Models](https://mlsanity.com/paper/2603.16445/) (2026)
- [DongYuan: An LLM-Based Framework for Integrative Chinese and Western Medicine Spleen-Stomach Disorders Diagnosis](https://mlsanity.com/paper/2603.28191/) (2026)
- [Revisiting the shutdown problem](https://mlsanity.com/paper/2606.08296/) (2026)
- [PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology](https://mlsanity.com/paper/2607.27378/) (2026)

## Abstract (arXiv)

The rapid evolution of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) has driven major gains in reasoning, perception, and generation across language and vision, yet whether these advances translate into comparable improvements in safety remains unclear, partly due to fragmented evaluations that focus on isolated modalities or threat models. In this report, we present an integrated safety evaluation of six frontier models--GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5--assessing each across language, vision-language, and image generation using a unified protocol that combines benchmark, adversarial, multilingual, and compliance evaluations. By aggregating results into safety leaderboards and model profiles, we reveal a highly uneven safety landscape: while GPT-5.2 demonstrates consistently strong and balanced performance, other models exhibit clear trade-offs across benchmark safety, adversarial robustness, multilingual generalization, and regulatory compliance. Despite strong results under standard benchmarks, all models remain highly vulnerable under adversarial testing, with worst-case safety rates dropping below 6%. Text-to-image models show slightly stronger alignment in regulated visual risk categories, yet remain fragile when faced with adversarial or semantically ambiguous prompts. Overall, these findings highlight that safety in frontier models is inherently multidimensional--shaped by modality, language, and evaluation design--underscoring the need for standardized, holistic safety assessments to better reflect real-world risk and guide responsible deployment.