mlsanityData to 14 Oct 2026

Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings

arXiv 2503.15092 ↗First posted 19 Mar 2025cs.CR cs.AI cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

This study presents the first comprehensive safety evaluation of the DeepSeek models, focusing on evaluating the safety risks associated with their generated content. Our evaluation encompasses DeepSeek's latest generation of large language models, multimodal large language models, and text-to-image models, systematically examining their performance regarding unsafe content generation. Notably, we developed a bilingual (Chinese-English) safety evaluation dataset tailored to Chinese sociocultural contexts, enabling a more thorough evaluation of the safety capabilities of Chinese-developed models. Experimental results indicate that despite their strong general capabilities, DeepSeek models exhibit significant safety vulnerabilities across multiple risk dimensions, including algorithmic discrimination and sexual content. These findings provide crucial insights for understanding and improving the safety of large foundation models. Our code is available at https://github.com/NY1024/DeepSeek-Safety-Eval.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 36 of its 48 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

36 of 48 references.

References · 36

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  5. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  6. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  7. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
  8. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 274
  9. 2024A Survey on Mixture of Experts in Large Language Modelscited by 237
  10. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  11. 2022DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Modelscited by 168
  12. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 135
24 more
  1. 2021Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical Worldcited by 127
  2. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 109
  3. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1cited by 91
  4. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 79
  5. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  6. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 72
  7. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
  8. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 64
  9. 2019Training Robust Deep Neural Networks via Adversarial Noise Propagationcited by 62
  10. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 59
  11. 2021A Comprehensive Evaluation Framework for Deep Model Robustnesscited by 42
  12. 2021Harnessing Perceptual Adversarial Patches for Crowd Countingcited by 39
  13. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
  14. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Modelscited by 33
  15. 2024Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Modelscited by 27
  16. 2024Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?cited by 25
  17. 2025o3-mini vs DeepSeek-R1: Which One is Safer?cited by 24
  18. 2024Automatic Jailbreaking of the Text-to-Image Generative AI Systemscited by 23
  19. 2024HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Modelscited by 16
  20. 2025CogMorph: Cognitive Morphing Attacks for Text-to-Image Modelscited by 12
  21. 2025Challenges in Ensuring AI Safety in DeepSeek-R1 Models: The Shortcomings of Reinforcement Learning Strategiescited by 12
  22. 2025The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled modelscited by 12
  23. 2024DLP: towards active defense against backdoor attacks with decoupled learning processcited by 10
  24. 2024NBA: defensive distillation for backdoor removal via neural behavior alignmentcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 8.

27 papers.

Sort:

By area:

  1. 2025Safety in Large Reasoning Models: A Surveycited by 64
  2. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 44
  3. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
  4. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Modelscited by 29
  5. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  6. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
  7. 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 20
  8. 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Modelscited by 13
  9. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 12
  10. 2025Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructionscited by 11
  11. 2026What Matters For Safety Alignment?cited by 7
  12. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
15 more
  1. 2025Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?cited by 6
  2. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  3. 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledgecited by 4
  4. 2025PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreakingcited by 3
  5. 2025The Trojan Example: Jailbreaking LLMs through Template Filling and Unsafety Reasoningcited by 3
  6. 2025PRJ: Perception-Retrieval-Judgement for Generated Imagescited by 2
  7. 2025Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytellingcited by 2
  8. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2
  9. 2026Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learningcited by 2
  10. 2025RRTL: Red Teaming Reasoning Large Language Models in Tool Learningcited by 1
  11. 2025HauntAttack: When Attack Follows Reasoning as a Shadowcited by 1
  12. 2026Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Modelscited by 0
  13. 2026Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framingcited by 0
  14. 2026JT-SAFE-V2: Safety-by-Design Foundation Model with World-Context Datacited by 0
  15. 2026SoK: Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Safety Assessment of Chinese Large Language Models
  2. 2025Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts
  3. 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models
  4. 2025Safety Evaluation of DeepSeek Models in Chinese Contexts
  5. 2026A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5
  6. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
  7. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
  8. 2026KSAFE-MM: A Multimodal Safety Benchmark via Localized Contextualization for Korean Cultural Risks