mlsanityData to 14 Oct 2026

General-Purpose Retrieval-Enhanced Medical Prediction Model Using Near-Infinite History

arXiv 2310.20204 ↗First posted 31 Oct 2023cs.LG cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) has recently shown promising results in medical predictions using electronic health records (EHRs). However, since ML models typically have a limited capability in terms of input sizes, selecting specific medical events from EHRs for use as input is necessary. This selection process, often relying on expert opinion, can cause bottlenecks in development. We propose Retrieval-Enhanced Medical prediction model (REMed) to address such challenges. REMed can essentially evaluate unlimited medical events, select the relevant ones, and make predictions. This allows for an unrestricted input size, eliminating the need for manual event selection. We verified these properties through experiments involving 27 clinical prediction tasks across four independent cohorts, where REMed outperformed the baselines. Notably, we found that the preferences of REMed align closely with those of medical experts. We expect our approach to significantly expedite the development of EHR prediction models by minimizing clinicians' need for manual involvement.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

25 of 40 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  8. 2020Rethinking Attention with Performerscited by 1,601
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  10. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  11. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  12. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 349
13 more
  1. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  2. 2022Recurrent Memory Transformercited by 203
  3. 2022Mega: Moving Average Equipped Gated Attentioncited by 179
  4. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
  5. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 98
  6. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
  7. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 36
  8. 2021Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embeddingcited by 24
  9. 2021Literature-Augmented Clinical Outcome Predictioncited by 20
  10. 2022UniHPF : Universal Healthcare Predictive Framework with Zero Domain Knowledgecited by 4
  11. 2023Conceptualizing Machine Learning for Dynamic Information Retrieval of Electronic Health Record Notescited by 4
  12. 2018Embedding Electronic Health Records for Clinical Information Retrievalcited by 3
  13. 2020All Data Inclusive, Deep Learning Models to Predict Critical Events in the Medical Information Mart for Intensive Care III Database (MIMIC III)cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 5.

12 papers.

  1. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  2. 2025A Collection of Innovations in Medical AI for patient records in 2024cited by 7
  3. 2025LabTOP: A Unified Model for Lab Test Outcome Prediction on Electronic Health Recordscited by 5
  4. 2025EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Recordscited by 3
  5. 2025Time-Aware Attention for Enhanced Electronic Health Records Modelingcited by 3
  6. 2024Towards Predicting Temporal Changes in a Patient's Chest X-ray Images based on Electronic Health Recordscited by 2
  7. 2026Multi-lingual Multi-institutional Electronic Health Record based Predictive Modelcited by 1
  8. 2024LML-DAP: Language Model Learning a Dataset for Data-Augmented Predictioncited by 0
  9. 2026ReToP: Learning to Rewrite Electronic Health Records for Clinical Predictioncited by 0
  10. 2026EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Recordscited by 0
  11. 2026Learning to Select Source-Traceable Evidence for Language-Model Prediction from Irregular Clinical Time Seriescited by 0
  12. 2026EviGen: Predictive Evidence Scaffolding for Verifiable Clinical Rationale Generationcited by 0