# Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings

arXiv: https://arxiv.org/abs/2503.15092 · HTML: https://mlsanity.com/paper/2503.15092/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 48 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 30 of 36)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023)
- [A Survey on Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2407.06204/) (2024)
- [JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2404.03027/) (2024)
- [DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models](https://mlsanity.com/paper/2210.06998/) (2022)
- [Jailbreaking Attack against Multimodal Large Language Model](https://mlsanity.com/paper/2402.02309/) (2024)
- [Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical World](https://mlsanity.com/paper/2103.01050/) (2021)
- [Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt](https://mlsanity.com/paper/2406.04031/) (2024)
- [The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1](https://mlsanity.com/paper/2502.12659/) (2025)
- [RobustART: Benchmarking Robustness on Architecture Design and Training Techniques](https://mlsanity.com/paper/2109.05211/) (2021)
- [SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety](https://mlsanity.com/paper/2404.05399/) (2024)
- [Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models](https://mlsanity.com/paper/2404.02928/) (2024)
- [SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models](https://mlsanity.com/paper/2410.18927/) (2024)
- [Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models](https://mlsanity.com/paper/2502.11054/) (2025)
- [Training Robust Deep Neural Networks via Adversarial Noise Propagation](https://mlsanity.com/paper/1909.09034/) (2019)
- [X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection](https://mlsanity.com/paper/2302.09491/) (2023)
- [A Comprehensive Evaluation Framework for Deep Model Robustness](https://mlsanity.com/paper/2101.09617/) (2021)
- [Harnessing Perceptual Adversarial Patches for Crowd Counting](https://mlsanity.com/paper/2109.07986/) (2021)
- [Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks](https://mlsanity.com/paper/2406.06302/) (2024)
- [S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models](https://mlsanity.com/paper/2405.14191/) (2024)
- [Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models](https://mlsanity.com/paper/2408.00523/) (2024)
- [Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?](https://mlsanity.com/paper/2406.10486/) (2024)
- [o3-mini vs DeepSeek-R1: Which One is Safer?](https://mlsanity.com/paper/2501.18438/) (2025)
- [Automatic Jailbreaking of the Text-to-Image Generative AI Systems](https://mlsanity.com/paper/2405.16567/) (2024)

## Built on by (most cited first, 27 of 27)

- [Safety in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2504.17704/) (2025)
- [Manipulating Multimodal Agents via Cross-Modal Prompt Injection](https://mlsanity.com/paper/2504.14348/) (2025)
- [AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions](https://mlsanity.com/paper/2506.14697/) (2025)
- [SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models](https://mlsanity.com/paper/2504.08813/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment](https://mlsanity.com/paper/2505.14667/) (2025)
- [Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw](https://mlsanity.com/paper/2603.12644/) (2026)
- [VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models](https://mlsanity.com/paper/2505.19684/) (2025)
- [Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025](https://mlsanity.com/paper/2506.12430/) (2025)
- [Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructions](https://mlsanity.com/paper/2509.01444/) (2025)
- [What Matters For Safety Alignment?](https://mlsanity.com/paper/2601.03868/) (2026)
- [POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models](https://mlsanity.com/paper/2505.06579/) (2025)
- [Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?](https://mlsanity.com/paper/2509.06350/) (2025)
- [Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures](https://mlsanity.com/paper/2601.19928/) (2026)
- [SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledge](https://mlsanity.com/paper/2511.13127/) (2025)
- [PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking](https://mlsanity.com/paper/2507.21540/) (2025)
- [The Trojan Example: Jailbreaking LLMs through Template Filling and Unsafety Reasoning](https://mlsanity.com/paper/2510.21190/) (2025)
- [PRJ: Perception-Retrieval-Judgement for Generated Images](https://mlsanity.com/paper/2506.03683/) (2025)
- [Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytelling](https://mlsanity.com/paper/2510.15068/) (2025)
- [Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability](https://mlsanity.com/paper/2512.01848/) (2025)
- [Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning](https://mlsanity.com/paper/2602.13562/) (2026)
- [RRTL: Red Teaming Reasoning Large Language Models in Tool Learning](https://mlsanity.com/paper/2505.17106/) (2025)
- [HauntAttack: When Attack Follows Reasoning as a Shadow](https://mlsanity.com/paper/2506.07031/) (2025)
- [Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models](https://mlsanity.com/paper/2603.09246/) (2026)
- [Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framing](https://mlsanity.com/paper/2604.15725/) (2026)
- [JT-SAFE-V2: Safety-by-Design Foundation Model with World-Context Data](https://mlsanity.com/paper/2605.24414/) (2026)
- [SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes](https://mlsanity.com/paper/2609.19705/) (2026)

## Abstract (arXiv)

This study presents the first comprehensive safety evaluation of the DeepSeek models, focusing on evaluating the safety risks associated with their generated content. Our evaluation encompasses DeepSeek's latest generation of large language models, multimodal large language models, and text-to-image models, systematically examining their performance regarding unsafe content generation. Notably, we developed a bilingual (Chinese-English) safety evaluation dataset tailored to Chinese sociocultural contexts, enabling a more thorough evaluation of the safety capabilities of Chinese-developed models. Experimental results indicate that despite their strong general capabilities, DeepSeek models exhibit significant safety vulnerabilities across multiple risk dimensions, including algorithmic discrimination and sexual content. These findings provide crucial insights for understanding and improving the safety of large foundation models. Our code is available at https://github.com/NY1024/DeepSeek-Safety-Eval.