# DDXPlus: A New Dataset For Automatic Medical Diagnosis

arXiv: https://arxiv.org/abs/2205.09148 · HTML: https://mlsanity.com/paper/2205.09148/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 59 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 6 of 6)

- [End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis](https://mlsanity.com/paper/1901.10623/) (2019)
- [Diaformer: Automatic Diagnosis via Symptoms Sequence Generation](https://mlsanity.com/paper/2112.10433/) (2021)
- [Efficient Symptom Inquiring and Diagnosis via Adaptive Alignment of Reinforcement Learning and Classification](https://mlsanity.com/paper/2112.00733/) (2021)
- ["My nose is running.""Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics](https://mlsanity.com/paper/2204.13953/) (2022)
- [Knowledge Grounded Conversational Symptom Detection with Graph Memory Networks](https://mlsanity.com/paper/2101.09773/) (2021)
- [A Bayesian Approach for Medical Inquiry and Disease Inference in Automated Differential Diagnosis](https://mlsanity.com/paper/2110.08393/) (2021)

## Built on by (most cited first, 30 of 88)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models](https://mlsanity.com/paper/2408.02442/) (2024)
- [MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot](https://mlsanity.com/paper/2502.04413/) (2025)
- [Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application](https://mlsanity.com/paper/2407.01885/) (2024)
- [StreamBench: Towards Benchmarking Continuous Improvement of Language Agents](https://mlsanity.com/paper/2406.08747/) (2024)
- [RareBench: Can LLMs Serve as Rare Diseases Specialists?](https://mlsanity.com/paper/2402.06341/) (2024)
- [CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis](https://mlsanity.com/paper/2407.13301/) (2024)
- [Towards Evaluating and Building Versatile Large Language Models for Medicine](https://mlsanity.com/paper/2408.12547/) (2024)
- [LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluation](https://mlsanity.com/paper/2305.13614/) (2023)
- [A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?](https://mlsanity.com/paper/2409.15277/) (2024)
- [TABLET: Learning From Instructions For Tabular Data](https://mlsanity.com/paper/2304.13188/) (2023)
- [PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration](https://mlsanity.com/paper/2406.01394/) (2024)
- [MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis](https://mlsanity.com/paper/2502.19175/) (2025)
- [MediTOD: An English Dialogue Dataset for Medical History Taking with Comprehensive Annotations](https://mlsanity.com/paper/2410.14204/) (2024)
- [Automating Expert-Level Medical Reasoning Evaluation of Large Language Models](https://mlsanity.com/paper/2507.07988/) (2025)
- [Assessing Adversarial Robustness of Large Language Models: An Empirical Study](https://mlsanity.com/paper/2405.02764/) (2024)
- [Large Language Models Perform Diagnostic Reasoning](https://mlsanity.com/paper/2307.08922/) (2023)
- [ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs](https://mlsanity.com/paper/2411.13547/) (2024)
- [EvoRoute: Experience-Driven Self-Routing LLM Agent Systems](https://mlsanity.com/paper/2601.02695/) (2026)
- [Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applications](https://mlsanity.com/paper/2501.02460/) (2025)
- [Follow-up Question Generation For Enhanced Patient-Provider Conversations](https://mlsanity.com/paper/2503.17509/) (2025)
- [Fill In The Gaps: Model Calibration and Generalization with Synthetic Data](https://mlsanity.com/paper/2410.10864/) (2024)
- [Human-AI Collaborative Uncertainty Quantification](https://mlsanity.com/paper/2510.23476/) (2025)
- [Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance](https://mlsanity.com/paper/2502.13321/) (2025)
- [Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models](https://mlsanity.com/paper/2603.24844/) (2026)
- [Towards Trustworthy Automatic Diagnosis Systems by Emulating Doctors' Reasoning with Deep Reinforcement Learning](https://mlsanity.com/paper/2210.07198/) (2022)

## Abstract (arXiv)

There has been a rapidly growing interest in Automatic Symptom Detection (ASD) and Automatic Diagnosis (AD) systems in the machine learning research literature, aiming to assist doctors in telemedicine services. These systems are designed to interact with patients, collect evidence about their symptoms and relevant antecedents, and possibly make predictions about the underlying diseases. Doctors would review the interactions, including the evidence and the predictions, collect if necessary additional information from patients, before deciding on next steps. Despite recent progress in this area, an important piece of doctors' interactions with patients is missing in the design of these systems, namely the differential diagnosis. Its absence is largely due to the lack of datasets that include such information for models to train on. In this work, we present a large-scale synthetic dataset of roughly 1.3 million patients that includes a differential diagnosis, along with the ground truth pathology, symptoms and antecedents for each patient. Unlike existing datasets which only contain binary symptoms and antecedents, this dataset also contains categorical and multi-choice symptoms and antecedents useful for efficient data collection. Moreover, some symptoms are organized in a hierarchy, making it possible to design systems able to interact with patients in a logical way. As a proof-of-concept, we extend two existing AD and ASD systems to incorporate the differential diagnosis, and provide empirical evidence that using differentials as training signals is essential for the efficiency of such systems or for helping doctors better understand the reasoning of those systems.