mlsanityData to 7 Oct 2026

Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints

arXiv 2604.01841 ↗First posted 2 Apr 2026cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Clinical prediction from structured electronic health records (EHRs) is challenging due to high dimensionality, heterogeneity, class imbalance, and distribution shift. While tabular in-context learning (TICL) and retrieval-augmented methods perform well on generic benchmarks, their behavior in clinical settings remains unclear. We present a multi-cohort EHR benchmark comparing classical, deep tabular, and TICL models across varying data scale, feature dimensionality, outcome rarity, and cross-cohort generalization. PFN-based TICL models are sample-efficient in low-data regimes but degrade under naive distance-based retrieval as heterogeneity and imbalance increase. We propose AWARE, a task-aligned retrieval framework using supervised embedding learning and lightweight adapters. AWARE improves AUPRC by up to 12.2% under extreme imbalance, with gains increasing with data complexity. Our results identify retrieval quality and retrieval-inference alignment as key bottlenecks for deploying tabular in-context learning in clinical prediction.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 49 of its 104 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

49 of 104 references.

References · 49

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  4. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  5. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  6. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  7. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  8. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  9. 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
  10. 2021Transformers Can Do Bayesian Inferencecited by 338
  11. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  12. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 267
37 more
  1. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  2. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  3. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  4. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  5. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 152
  6. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 145
  7. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 105
  8. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  9. 2015Learning to Diagnose with LSTM Recurrent Neural Networkscited by 99
  10. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 97
  11. 2019Analyzing and Improving Representations with the Soft Nearest Neighbor Losscited by 95
  12. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 74
  13. 2024Retrieval & Fine-Tuning for In-Context Tabular Modelscited by 58
  14. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  15. 2021DANets: Deep Abstract Networks for Tabular Data Classification and Regressioncited by 50
  16. 2022An Extensive Data Processing Pipeline for MIMIC-IVcited by 39
  17. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  18. 2023Trompt: Towards a Better Deep Neural Network for Tabular Datacited by 36
  19. 2025Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Datacited by 36
  20. 2023CPLLM: Clinical Prediction with Large Language Modelscited by 34
  21. 2023ExcelFormer: A neural network surpassing GBDTs on tabular datacited by 33
  22. 2024Mixture of In-Context Prompters for Tabular PFNscited by 32
  23. 2024EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Recordscited by 31
  24. 2024Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Latercited by 28
  25. 2024Emergency Department Decision Support using Clinical Pseudo-notescited by 22
  26. 2024Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Datacited by 22
  27. 2021Retrieval & Interaction Machine for Tabular Data Predictioncited by 21
  28. 2024CORE-BEHRT: A Carefully Optimized and Rigorously Evaluated BEHRTcited by 20
  29. 2024IM-Context: In-Context Learning for Imbalanced Regression Taskscited by 7
  30. 2025Scalable In-Context Learning on Tabular Data via Retrieval-Augmented Large Language Modelscited by 7
  31. 2023Clinical Risk Prediction Using Language Models: Benefits And Considerationscited by 6
  32. 2025Bridging Data Gaps of Rare Conditions in ICU: A Multi-Disease Adaptation Approach for Clinical Predictioncited by 4
  33. 2025CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Recordscited by 4
  34. 2025CardioTabNet: A Novel Hybrid Transformer Model for Heart Disease Prediction using Tabular Medical Datacited by 3
  35. 2025Longitudinal Progression Prediction of Alzheimer's Disease with Tabular Foundation Modelcited by 3
  36. 2025Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHRcited by 2
  37. 2025Explainable AI for Infection Prevention and Control: Modeling CPE Acquisition and Patient Outcomes in an Irish Hospital with Transformerscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EXAONE Tabular 1.0 : Technical Reportcited by 7
  2. 2026Tabular Foundation Models for Clinical Survival Analysis via Survival-Aware Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EHR-RAG: Bridging Long-Horizon Structured Electronic Health Records and Large Language Models via Enhanced Retrieval-Augmented Generation
  2. 2026EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records
  3. 2026Lightweight Retrieval-Augmented Generation and Large Language Model-Based Modeling for Scalable Patient-Trial Matching
  4. 2024Lessons Learned on Information Retrieval in Electronic Health Records: A Comparison of Embedding Models and Pooling Strategies
  5. 2024EMERGE: Enhancing Multimodal Electronic Health Records Predictive Modeling with Retrieval-Augmented Generation
  6. 2025EHRStruct: A Comprehensive Benchmark Framework for Evaluating Large Language Models on Structured Electronic Health Record Tasks
  7. 2026Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning
  8. 2025Retrieval-augmented in-context learning for multimodal large language models in disease classification