From Basic to Extra Features: Hypergraph Transformer Pretrain-then-Finetuning for Balanced Clinical Predictions on EHR
Not on the map. Why?
Abstract (arXiv)
Electronic Health Records (EHRs) contain rich patient information and are crucial for clinical research and practice. In recent years, deep learning models have been applied to EHRs, but they often rely on massive features, which may not be readily available for all patients. We propose HTP-Star, which leverages hypergraph structures with a pretrain-then-finetune framework for modeling EHR data, enabling seamless integration of additional features. Additionally, we design two techniques, namely (1) Smoothness-inducing Regularization and (2) Group-balanced Reweighting, to enhance the model's robustness during fine-tuning. Through experiments conducted on two real EHR datasets, we demonstrate that HTP-Star consistently outperforms various baselines while striking a balance between patients with basic and extra features.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 58 references that are in the map. First posted on arXiv 9 Jun 2024.
Builds on
29 of 58 references.
References · 29
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2018On First-Order Meta-Learning Algorithmscited by 1,003
- 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
- 2016RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanismcited by 344
- 2019BEHRT: Transformer for Electronic Health Recordscited by 258
- 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
- 2019Pre-training of Graph Augmented Transformers for Medication Recommendationcited by 124
17 more
- 2019Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformercited by 88
- 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
- 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 75
- 2019Graph Transfer Learning via Adversarial Domain Adaptation with Graph Convolutioncited by 67
- 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
- 2021Adaptive Transfer Learning on Graph Neural Networkscited by 31
- 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
- 2023Learning Missing Modal Electronic Health Records with Unified Multi-modal Data Embedding and Modality-Aware Attentioncited by 25
- 2023Multimodal Pretraining of Medical Time Series and Notescited by 22
- 2024RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Recordscited by 21
- 2021Context-aware Health Event Prediction via Transition Functions on Dynamic Disease Graphscited by 20
- 2021Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosiscited by 18
- 2022Graph-Text Multi-Modal Pre-training for Medical Representation Learningcited by 10
- 2022Fair admission risk prediction with proportional multicalibrationcited by 8
- 2022Pretraining ECG Data with Adversarial Masking Improves Model Generalizability for Data-Scarce Taskscited by 3
- 2023Graph AI in Medicinecited by 3
- 2020CHEER: Rich Model Helps Poor Model via Knowledge Infusioncited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.