mlsanityData to 14 Oct 2026

ECG-Expert-QA: A Benchmark for Evaluating Medical Large Language Models in Heart Disease Diagnosis

arXiv 2502.17475 ↗First posted 16 Feb 2025eess.SP cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We present ECG-Expert-QA, a comprehensive multimodal dataset for evaluating diagnostic capabilities in electrocardiogram (ECG) interpretation. It combines real-world clinical ECG data with systematically generated synthetic cases, covering 12 essential diagnostic tasks and totaling 47,211 expert-validated QA pairs. These encompass diverse clinical scenarios, from basic rhythm recognition to complex diagnoses involving rare conditions and temporal changes. A key innovation is the support for multi-turn dialogues, enabling the development of conversational medical AI systems that emulate clinician-patient or interprofessional interactions. This allows for more realistic assessment of AI models' clinical reasoning, diagnostic accuracy, and knowledge integration. Constructed through a knowledge-guided framework with strict quality control, ECG-Expert-QA ensures linguistic and clinical consistency, making it a high-quality resource for advancing AI-assisted ECG interpretation. It challenges models with tasks like identifying subtle ischemic changes and interpreting complex arrhythmias in context-rich scenarios. To promote research transparency and collaboration, the dataset, accompanying code, and prompts are publicly released at https://github.com/Zaozzz/ECG-Expert-QA

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 27 of its 55 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

27 of 55 references.

References · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2020Learning to summarize from human feedbackcited by 2,588
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  6. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  7. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  8. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
  9. 2021Pretrained Transformers as Universal Computation Enginescited by 162
  10. 2017What do we need to build explainable AI systems for the medical domain?cited by 129
  11. 2023ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogramcited by 54
  12. 2024ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosiscited by 38
15 more
  1. 2024SPRIG: Improving Large Language Model Performance by System Prompt Optimizationcited by 29
  2. 2018ECG arrhythmia classification using a 2-D convolutional neural networkcited by 17
  3. 2024MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skillscited by 17
  4. 2023An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Modelscited by 13
  5. 2024Electrocardiogram Report Generation and Question Answering via Retrieval-Augmented Self-Supervised Modelingcited by 7
  6. 2024Thinking Before Speaking: A Role-playing Model with Mindsetcited by 5
  7. 2023Transforming ECG Diagnosis:An In-depth Review of Transformer-based DeepLearning Models in Cardiovascular Disease Detectioncited by 4
  8. 2023MSW-Transformer: Multi-Scale Shifted Windows Transformer Networks for 12-Lead ECG Classificationcited by 4
  9. 2017Detection of Inferior Myocardial Infarction using Shallow Convolutional Neural Networkscited by 3
  10. 2018Robust Heartbeat Detection from Multimodal Data via CNN-based Generalizable Information Fusioncited by 3
  11. 2022HeartBEiT: Vision Transformer for Electrocardiogram Data Improves Diagnostic Performance at Low Sample Sizescited by 3
  12. 2024MoRE: Multi-Modal Contrastive Pre-training with Transformers on X-Rays, ECGs, and Diagnostic Reportcited by 3
  13. 2023MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notescited by 2
  14. 2018Time series kernel similarities for predicting Paroxysmal Atrial Fibrillation from ECGscited by 1
  15. 2018Predictive Modeling of Biomedical Signals Using Controlled Spatial Transformationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scalecited by 6
  2. 2026CARE-ECG: Causal Agent-based Reasoning for Explainable and Counterfactual ECG Interpretationcited by 2
  3. 2026VitalAgent: A Tool-Augmented Agent for Reactive and Proactive Physiological Monitoring over Wearable Health Datacited by 0
  4. 2026MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domaincited by 0
  5. 2026Large language models in medical time series analysiscited by 0
  6. 2026ECG-RSI: Recursive Self-Improvement through Targeted Electrocardiogram Waveform Evolutioncited by 0