# o3-mini vs DeepSeek-R1: Which One is Safer?

arXiv: https://arxiv.org/abs/2501.18438 · HTML: https://mlsanity.com/paper/2501.18438/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 22 of 22)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation](https://mlsanity.com/paper/2310.06987/) (2023)
- [A StrongREJECT for Empty Jailbreaks](https://mlsanity.com/paper/2402.10260/) (2024)
- [SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal](https://mlsanity.com/paper/2406.14598/) (2024)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2402.05044/) (2024)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models](https://mlsanity.com/paper/2311.08370/) (2023)
- [SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models](https://mlsanity.com/paper/2410.18927/) (2024)
- [ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors](https://mlsanity.com/paper/2402.16444/) (2024)
- [S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models](https://mlsanity.com/paper/2405.14191/) (2024)
- [WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models](https://mlsanity.com/paper/2408.03837/) (2024)
- [CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2406.10311/) (2024)
- [Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation](https://mlsanity.com/paper/2501.17749/) (2025)
- [LongSafety: Enhance Safety for Long-Context LLMs](https://mlsanity.com/paper/2411.06899/) (2024)
- [ASTRAL: Automated Safety Testing of Large Language Models](https://mlsanity.com/paper/2501.17132/) (2025)

## Built on by (most cited first, 24 of 24)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Safety in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2504.17704/) (2025)
- [100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models](https://mlsanity.com/paper/2505.00551/) (2025)
- [SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models](https://mlsanity.com/paper/2504.08813/) (2025)
- [Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings](https://mlsanity.com/paper/2503.15092/) (2025)
- [To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models](https://mlsanity.com/paper/2502.12202/) (2025)
- [Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities](https://mlsanity.com/paper/2503.17979/) (2025)
- [A Comparison of DeepSeek and Other LLMs](https://mlsanity.com/paper/2502.03688/) (2025)
- [Safety Evaluation of DeepSeek Models in Chinese Contexts](https://mlsanity.com/paper/2502.11137/) (2025)
- [Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking?](https://mlsanity.com/paper/2505.17650/) (2025)
- [Thinking Preference Optimization](https://mlsanity.com/paper/2502.13173/) (2025)
- [Dynamic Chain-of-Thought: Towards Adaptive Deep Reasoning](https://mlsanity.com/paper/2502.10428/) (2025)
- [When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoning](https://mlsanity.com/paper/2602.01365/) (2026)
- [Policy Frameworks for Transparent Chain-of-Thought Reasoning in Large Language Models](https://mlsanity.com/paper/2503.14521/) (2025)
- [Is Reasoning All You Need? Probing Bias in the Age of Reasoning Language Models](https://mlsanity.com/paper/2507.02799/) (2025)
- [AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models](https://mlsanity.com/paper/2509.24269/) (2025)
- [LLMs in Disease Diagnosis: A Comparative Study of DeepSeek-R1 and O3 Mini Across Chronic Health Conditions](https://mlsanity.com/paper/2503.10486/) (2025)
- [Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy](https://mlsanity.com/paper/2507.01327/) (2025)
- [How Does the Thinking Step Influence Model Safety? An Entropy-based Safety Reminder for LRMs](https://mlsanity.com/paper/2601.03662/) (2026)
- [Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability](https://mlsanity.com/paper/2512.01848/) (2025)
- [RRTL: Red Teaming Reasoning Large Language Models in Tool Learning](https://mlsanity.com/paper/2505.17106/) (2025)
- [Analysing Safety Risks in LLMs Fine-Tuned with Pseudo-Malicious Cyber Security Data](https://mlsanity.com/paper/2505.09974/) (2025)
- [Efficient Strategy for Improving Large Language Model (LLM) Capabilities](https://mlsanity.com/paper/2508.04073/) (2025)
- [RAJ-PGA: Reasoning-Activated Jailbreak and Principle-Guided Alignment Framework for Large Reasoning Models](https://mlsanity.com/paper/2508.12897/) (2025)

## Abstract (arXiv)

The irruption of DeepSeek-R1 constitutes a turning point for the AI industry in general and the LLMs in particular. Its capabilities have demonstrated outstanding performance in several tasks, including creative thinking, code generation, maths and automated program repair, at apparently lower execution cost. However, LLMs must adhere to an important qualitative property, i.e., their alignment with safety and human values. A clear competitor of DeepSeek-R1 is its American counterpart, OpenAI's o3-mini model, which is expected to set high standards in terms of performance, safety and cost. In this technical report, we systematically assess the safety level of both DeepSeek-R1 (70b version) and OpenAI's o3-mini (beta version). To this end, we make use of our recently released automated safety testing tool, named ASTRAL. By leveraging this tool, we automatically and systematically generated and executed 1,260 test inputs on both models. After conducting a semi-automated assessment of the outcomes provided by both LLMs, the results indicate that DeepSeek-R1 produces significantly more unsafe responses (12%) than OpenAI's o3-mini (1.2%).