mlsanityData to 7 Oct 2026

A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasets

arXiv 2204.08997 ↗First posted 19 Apr 2022cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

In recent years, interest has arisen in using machine learning to improve the efficiency of automatic medical consultation and enhance patient experience. In this article, we propose two frameworks to support automatic medical consultation, namely doctor-patient dialogue understanding and task-oriented interaction. We create a new large medical dialogue dataset with multi-level finegrained annotations and establish five independent tasks, including named entity recognition, dialogue act classification, symptom label inference, medical report generation and diagnosis-oriented dialogue policy. We report a set of benchmark results for each task, which shows the usability of the dataset and sets a baseline for future studies. Both code and data is available from https://github.com/lemuria-wchen/imcs21.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 21 of its 56 references that are in the map. First posted on arXiv 19 Apr 2022.

Builds on

21 of 56 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  5. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  6. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  7. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  8. 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 157
  9. 2016Recurrent Neural Network for Text Classification with Multi-Task Learningcited by 144
  10. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
  11. 2018Chinese NER Using Lattice LSTMcited by 81
  12. 2019End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosiscited by 70
9 more
  1. 2020Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structurescited by 49
  2. 2020MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generationcited by 45
  3. 2020Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognitioncited by 35
  4. 2021Lexicon Enhanced Chinese Sequence Labeling Using BERT Adaptercited by 34
  5. 2021ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generationcited by 31
  6. 2021Semi-Supervised Variational Reasoning for Medical Dialogue Generationcited by 30
  7. 2020Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generationcited by 24
  8. 2019Extracting Symptoms and their Status from Clinical Conversationscited by 23
  9. 2021Drug Package Recommendation via Interaction-aware Graph Inductioncited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 17; 2025: 12; 2026: 7.

45 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  2. 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulatorcited by 99
  3. 2023DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultationcited by 89
  4. 2023BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPTcited by 64
  5. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  6. 2024Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulatorcited by 36
  7. 2025Real-World Doctor Agent with Proactive Consultation through Multi-Agent Reinforcement Learningcited by 26
  8. 2024MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integrationcited by 21
  9. 2024MediTOD: An English Dialogue Dataset for Medical History Taking with Comprehensive Annotationscited by 21
  10. 2024Health-LLM: Personalized Retrieval-Augmented Disease Prediction Systemcited by 17
  11. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  12. 2024Large Language Model Benchmarks in Medical Taskscited by 16
33 more
  1. 20253MDBench: Medical Multimodal Multi-agent Dialogue Benchmarkcited by 16
  2. 2023Overview of the PromptCBLUE Shared Task in CHIP2023cited by 14
  3. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 14
  4. 2023MidMed: Towards Mixed-Type Dialogues for Medical Consultationcited by 13
  5. 2025Towards Online Multi-Modal Social Interaction Understandingcited by 12
  6. 2025InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Trainingcited by 11
  7. 2022DxFormer: A Decoupled Automatic Diagnostic System Based on Decoder-Encoder Transformer with Dense Symptom Representationscited by 10
  8. 2024Integrating Physician Diagnostic Logic into Large Language Models: Preference Learning from Process Feedbackcited by 9
  9. 2023PlugMed: Improving Specificity in Patient-Centered Medical Dialogue Generation using In-Context Learningcited by 8
  10. 2024PIORS: Personalized Intelligent Outpatient Reception based on Large Language Model with Multi-Agents Medical Scenario Simulationcited by 8
  11. 2023KNSE: A Knowledge-aware Natural Language Inference Framework for Dialogue Symptom Status Recognitioncited by 5
  12. 2024Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challengescited by 5
  13. 2024CauESC: A Causal Aware Model for Emotional Support Conversationcited by 4
  14. 2024Conversational Disease Diagnosis via External Planner-Controlled Large Language Modelscited by 3
  15. 2025Improving Interactive Diagnostic Ability of a Large Language Model Agent Through Clinical Experience Learningcited by 3
  16. 2026Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultationscited by 3
  17. 2026Omni-MMSI: Toward Identity-attributed Social Interaction Understandingcited by 3
  18. 2024Medical Dialogue Generation via Intuitive-then-Analytical Differential Diagnosiscited by 2
  19. 2024CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinationscited by 2
  20. 2024Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clusteringcited by 1
  21. 2024Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignmentcited by 1
  22. 2025A Typology of Synthetic Datasets for Dialogue Processing in Clinical Contextscited by 1
  23. 2025CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operationcited by 1
  24. 2025ZhiFangDanTai: Fine-tuning Graph-based Retrieval-Augmented Generation Model for Traditional Chinese Medicine Formulacited by 1
  25. 2026MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reportscited by 1
  26. 2024HBot: A Chatbot for Healthcare Applications in Traditional Chinese Medicine Based on Human Body 3D Visualizationcited by 0
  27. 2025Towards Automatic Continual Learning: A Self-Adaptive Framework for Continual Instruction Tuningcited by 0
  28. 2025"Where does it hurt?" -- Dataset and Study on Physician Intent Trajectories in Doctor Patient Dialoguescited by 0
  29. 2025Text Mining Analysis of Symptom Patterns in Medical Chatbot Conversationscited by 0
  30. 2026Evidence-Based Intelligent Diagnostic and Therapeutic Visualization System with Large Language Models: Multi-Turn Interaction and Multimodal Treatment Plan Generationcited by 0
  31. 2026ELICITED: EHR-grounded Longitudinal Interactive Conversations for Information-seeking Triage Evaluation and Decision-makingcited by 0
  32. 2026MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesiscited by 0
  33. 2026MedRoundsQA: A Persona and Difficulty Aware Evaluation for Multi-Turn Medical Consultationscited by 0