mlsanityData to 14 Oct 2026

Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction

arXiv 2501.17326 ↗First posted 28 Jan 2025cs.CL cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Clinical diagnosis prediction models, when provided with a patient's medical history, aim to detect potential diseases early, facilitating timely intervention and improving prognostic outcomes. However, the inherent scarcity of patient data and large disease candidate space often pose challenges in developing satisfactory models for this intricate task. The exploration of leveraging Large Language Models (LLMs) for encapsulating clinical decision processes has been limited. We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes. Experimental results on MIMIC-III and IV datasets show that MERA achieves the state-of-the-art diagnosis prediction performance and dramatically elevates the diagnosis prediction capabilities of generative LMs.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

34 of 60 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  7. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 349
  8. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  9. 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 156
  10. 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 127
  11. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  12. 2017Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networkscited by 116
22 more
  1. 2023EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Modelscited by 105
  2. 2023PMC-LLaMA: Towards Building Open-source Language Models for Medicinecited by 88
  3. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 78
  4. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  5. 2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 74
  6. 2016Deepr: A Convolutional Net for Medical Recordscited by 60
  7. 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
  8. 2023A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnosticscited by 43
  9. 2023CPLLM: Clinical Prediction with Large Language Modelscited by 34
  10. 2021Collaborative Graph Learning with Auxiliary Text for Temporal Event Prediction in Healthcarecited by 30
  11. 2023DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patientscited by 30
  12. 2021Context-aware Health Event Prediction via Transition Functions on Dynamic Disease Graphscited by 20
  13. 2022DICE: Data-Efficient Clinical Event Extraction with Generative Modelscited by 20
  14. 2021HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learningcited by 14
  15. 2022Multimodal data matters: language model pre-training over structured and unstructured electronic health recordscited by 13
  16. 2022Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?cited by 13
  17. 2023Event-Based Contrastive Learning for Medical Time Seriescited by 13
  18. 2023ExBEHRT: Extended Transformer for Electronic Health Records to Predict Disease Subtypes & Progressionscited by 12
  19. 2023Hierarchical Pretraining on Multimodal Electronic Health Recordscited by 11
  20. 2024CliBench: A Multifaceted and Multigranular Evaluation of Large Language Models for Clinical Decision Makingcited by 8
  21. 2024CLIMB: A Benchmark of Clinical Bias in Large Language Modelscited by 7
  22. 2023STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 26
  2. 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Recordscited by 3
  3. 2025DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaborationcited by 2
  4. 2025RuCCoD: Towards Automated ICD Coding in Russiancited by 1
  5. 2026MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultationcited by 1
  6. 2025Stable Prediction of Adverse Events in Medical Time-Series Datacited by 0
  7. 2026Reinforcement Learning In Partially Verifiable Environments for Colon Cancer Treatment Planningcited by 0
  8. 2026Beyond ICD Codes: Semantic IDs for Generative Medical Codingcited by 0