mlsanityData to 14 Oct 2026

Temporal Fusion Nexus: A task-agnostic multi-modal embedding model for clinical narratives and irregular time series in post-kidney transplant care

arXiv 2601.08503 ↗First posted 13 Jan 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce Temporal Fusion Nexus (TFN), a multi-modal and task-agnostic embedding model to integrate irregular time series and unstructured clinical narratives. We analysed TFN in post-kidney transplant (KTx) care, with a retrospective cohort of 3382 patients, on three key outcomes: graft loss, graft rejection, and mortality. Compared to state-of-the-art model in post KTx care, TFN achieved higher performance for graft loss (AUC 0.96 vs. 0.94) and graft rejection (AUC 0.84 vs. 0.74). In mortality prediction, TFN yielded an AUC of 0.86. TFN outperformed unimodal baselines (approx 10% AUC improvement over time series only baseline, approx 5% AUC improvement over time series with static patient data). Integrating clinical text improved performance across all tasks. Disentanglement metrics confirmed robust and interpretable latent factors in the embedding space, and SHAP-based attributions confirmed alignment with clinical reasoning. TFN has potential application in clinical tasks beyond KTx, where heterogeneous data sources, irregular longitudinal data, and rich narrative documentation are available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 67 references that are in the map. First posted on arXiv 13 Jan 2026.

Builds on

35 of 67 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  6. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  7. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  9. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  10. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  11. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  12. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
23 more
  1. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  2. 2017Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networkscited by 116
  3. 2022Disentangled Representation Learningcited by 112
  4. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  5. 2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 74
  6. 2024DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistencycited by 53
  7. 2022A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Predictioncited by 48
  8. 2020Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Reviewcited by 47
  9. 2019Using Clinical Notes with Time Series Data for ICU Managementcited by 38
  10. 2021Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologiescited by 34
  11. 2022Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniquescited by 25
  12. 2022DCI-ES: An Extended Disentanglement Framework with Connections to Identifiabilitycited by 23
  13. 2018DeepSOFA: A Continuous Acuity Score for Critically Ill Patients using Clinically Interpretable Deep Learningcited by 22
  14. 2021Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detectioncited by 21
  15. 2022Multimodal data matters: language model pre-training over structured and unstructured electronic health recordscited by 13
  16. 2022On the Importance of Clinical Notes in Multi-modal Learning for EHR Datacited by 10
  17. 2022Review of Disentanglement Approaches for Medical Applications -- Towards Solving the Gordian Knot of Generative Models in Healthcarecited by 9
  18. 2023Labrador: Exploring the Limits of Masked Language Modeling for Laboratory Datacited by 9
  19. 2024Identifiability Guarantees for Causal Disentanglement from Purely Observational Datacited by 7
  20. 2024Triple Disentangled Representation Learning for Multimodal Affective Analysiscited by 4
  21. 2024Global Contrastive Training for Multimodal Electronic Health Records with Language Supervisioncited by 4
  22. 2024Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Recordscited by 3
  23. 2025Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical textscited by 1

Built on by

Not cited yet.