mlsanityData to 7 Oct 2026

MATA-Former & SIICU: Semantic Aware Temporal Alignment for High-Fidelity ICU Risk Prediction

arXiv 2604.01727 ↗First posted 2 Apr 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Forecasting evolving clinical risks relies on intrinsic pathological dependencies rather than mere chronological proximity, yet current methods struggle with coarse binary supervision and physical timestamps. To align predictive modeling with clinical logic, we propose the Medical-semantics Aware Time-ALiBi Transformer (MATA-Former), utilizing event semantics to dynamically parameterize attention weights to prioritize causal validity over time lags. Furthermore, we introduce Plateau-Gaussian Soft Labeling (PSL), reformulating binary classification into continuous multi-horizon regression for full-trajectory risk modeling. Evaluated on SIICU -- a newly constructed dataset featuring over 506k events with rigorous expert-verified, fine-grained annotations -- and the MIMIC-IV dataset, our framework demonstrates superior efficacy and robust generalization in capturing risks from text-intensive, irregular clinical time series.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 43 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

24 of 43 references.

References · 24

  1. 2020GLU Variants Improve Transformercited by 1,724
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  4. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  5. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  6. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  7. 2018Scalable and accurate deep learning for electronic health recordscited by 275
  8. 2019BEHRT: Transformer for Electronic Health Recordscited by 258
  9. 2019Time2Vec: Learning a Vector Representation of Timecited by 208
  10. 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 152
  11. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 145
  12. 2015Learning to Diagnose with LSTM Recurrent Neural Networkscited by 99
12 more
  1. 2023Do We Still Need Clinical Language Models?cited by 97
  2. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  3. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 75
  4. 2020A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Seriescited by 42
  5. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 25
  6. 2021Context-aware Health Event Prediction via Transition Functions on Dynamic Disease Graphscited by 20
  7. 2023Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Seriescited by 16
  8. 2025Multimodal Medical Code Tokenizercited by 14
  9. 2022Temporal Label Smoothing for Early Event Predictioncited by 12
  10. 2018Medical Concept Embedding with Time-Aware Attentioncited by 11
  11. 2025Time-Aware Attention for Enhanced Electronic Health Records Modelingcited by 3
  12. 2025Bi-Axial Transformers: Addressing the Increasing Complexity of EHR Classificationcited by 2

Built on by

Not cited yet.