# ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning

arXiv: https://arxiv.org/abs/2512.23440 · HTML: https://mlsanity.com/paper/2512.23440/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 29 Dec 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 16 of 16)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://mlsanity.com/paper/2402.14762/) (2024)
- [MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports](https://mlsanity.com/paper/2505.11733/) (2025)
- [Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases](https://mlsanity.com/paper/2503.04691/) (2025)
- [Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education](https://mlsanity.com/paper/2409.18924/) (2024)
- [Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models](https://mlsanity.com/paper/2510.10278/) (2025)
- [Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulators](https://mlsanity.com/paper/2501.09484/) (2025)
- [Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluation](https://mlsanity.com/paper/2510.09275/) (2025)
- [Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judges](https://mlsanity.com/paper/2508.00454/) (2025)
- [MedKGEval: A Knowledge Graph-Based Multi-Turn Evaluation Framework for Open-Ended Patient Interactions with Clinical LLMs](https://mlsanity.com/paper/2510.12224/) (2025)
- [SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models](https://mlsanity.com/paper/2505.15094/) (2025)

## Abstract (arXiv)

Clinical diagnosis begins with doctor-patient interaction, during which physicians iteratively gather information, determine examination and refine differential diagnosis through patients' response. This dynamic clinical-reasoning process is poorly represented by existing LLM benchmarks that focus on static question-answering. To mitigate these gaps, recent methods explore dynamic medical frameworks involving interactive clinical dialogues. Although effective, they often rely on limited, contamination-prone datasets and lack granular, multi-level evaluation. In this work, we propose ClinDEF, a dynamic framework for assessing clinical reasoning in LLMs through simulated diagnostic dialogues. Grounded in a disease knowledge graph, our method dynamically generates patient cases and facilitates multi-turn interactions between an LLM-based doctor and an automated patient agent. Our evaluation protocol goes beyond diagnostic accuracy by incorporating fine-grained efficiency analysis and rubric-based assessment of diagnostic quality. Experiments show that ClinDEF effectively exposes critical clinical reasoning gaps in state-of-the-art LLMs, offering a more nuanced and clinically meaningful evaluation paradigm.