Evidence-Grounded Subspecialty Reasoning: Evaluating a Curated Clinical Intelligence Layer on the 2025 Endocrinology Board-Style Examination
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
Background: Large language models have demonstrated strong performance on general medical examinations, but subspecialty clinical reasoning remains challenging due to rapidly evolving guidelines and nuanced evidence hierarchies. Methods: We evaluated January Mirror, an evidence-grounded clinical reasoning system, against frontier LLMs (GPT-5, GPT-5.2, Gemini-3-Pro) on a 120-question endocrinology board-style examination. Mirror integrates a curated endocrinology and cardiometabolic evidence corpus with a structured reasoning architecture to generate evidence-linked outputs. Mirror operated under a closed-evidence constraint without external retrieval. Comparator LLMs had real-time web access to guidelines and primary literature. Results: Mirror achieved 87.5% accuracy (105/120; 95% CI: 80.4-92.3%), exceeding a human reference of 62.3% and frontier LLMs including GPT-5.2 (74.6%), GPT-5 (74.0%), and Gemini-3-Pro (69.8%). On the 30 most difficult questions (human accuracy less than 50%), Mirror achieved 76.7% accuracy. Top-2 accuracy was 92.5% for Mirror versus 85.25% for GPT-5.2. Conclusions: Mirror provided evidence traceability: 74.2% of outputs cited at least one guideline-tier source, with 100% citation accuracy on manual verification. Curated evidence with explicit provenance can outperform unconstrained web retrieval for subspecialty clinical reasoning and supports auditability for clinical deployment.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 1 of its 15 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
1 of 15 references.
References · 1
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Scaling Clinical Judgment to Evaluate Medical AI
- 2026A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
- 2025Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases
- 2026Human-AI Co-reasoning for Clinical Diagnosis with Evidence-Integrated Language Agent
- 2025Measuring Stability Beyond Accuracy in Small Open-Source Medical Large Language Models for Pediatric Endocrinology
- 2025Building a Human-Verified Clinical Reasoning Dataset via a Human LLM Hybrid Pipeline for Trustworthy Medical AI
- 2026Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models
- 2026A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses