# LLMs in Disease Diagnosis: A Comparative Study of DeepSeek-R1 and O3 Mini Across Chronic Health Conditions

arXiv: https://arxiv.org/abs/2503.10486 · HTML: https://mlsanity.com/paper/2503.10486/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [RareBench: Can LLMs Serve as Rare Diseases Specialists?](https://mlsanity.com/paper/2402.06341/) (2024)
- [Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales](https://mlsanity.com/paper/2312.07399/) (2023)
- [o3-mini vs DeepSeek-R1: Which One is Safer?](https://mlsanity.com/paper/2501.18438/) (2025)
- [Probabilistic Medical Predictions of Large Language Models](https://mlsanity.com/paper/2408.11316/) (2024)
- [Large Language Models with Retrieval-Augmented Generation for Zero-Shot Disease Phenotyping](https://mlsanity.com/paper/2312.06457/) (2023)
- [Digital Diagnostics: The Potential Of Large Language Models In Recognizing Symptoms Of Common Illnesses](https://mlsanity.com/paper/2405.06712/) (2024)
- [Exploring Prompt Engineering: A Systematic Review with SWOT Analysis](https://mlsanity.com/paper/2410.12843/) (2024)
- [GPT-4's assessment of its performance in a USMLE-based case study](https://mlsanity.com/paper/2402.09654/) (2024)
- [Encouraging Responsible Use of Generative AI in Education: A Reward-Based Learning Approach](https://mlsanity.com/paper/2407.15022/) (2024)

## Built on by (most cited first, 3 of 3)

- [SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions](https://mlsanity.com/paper/2603.07379/) (2026)
- [LLM-Based Support for Diabetes Diagnosis: Opportunities, Scenarios, and Challenges with GPT-5](https://mlsanity.com/paper/2509.21450/) (2025)
- [Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches](https://mlsanity.com/paper/2606.01145/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are revolutionizing medical diagnostics by enhancing both disease classification and clinical decision-making. In this study, we evaluate the performance of two LLM- based diagnostic tools, DeepSeek R1 and O3 Mini, using a structured dataset of symptoms and diagnoses. We assessed their predictive accuracy at both the disease and category levels, as well as the reliability of their confidence scores. DeepSeek R1 achieved a disease-level accuracy of 76% and an overall accuracy of 82%, outperforming O3 Mini, which attained 72% and 75% respectively. Notably, DeepSeek R1 demonstrated exceptional performance in Mental Health, Neurological Disorders, and Oncology, where it reached 100% accuracy, while O3 Mini excelled in Autoimmune Disease classification with 100% accuracy. Both models, however, struggled with Respiratory Disease classification, recording accuracies of only 40% for DeepSeek R1 and 20% for O3 Mini. Additionally, the analysis of confidence scores revealed that DeepSeek R1 provided high-confidence predictions in 92% of cases, compared to 68% for O3 Mini. Ethical considerations regarding bias, model interpretability, and data privacy are also discussed to ensure the responsible integration of LLMs into clinical practice. Overall, our findings offer valuable insights into the strengths and limitations of LLM-based diagnostic systems and provide a roadmap for future enhancements in AI-driven healthcare.