# DT-BEHRT: Disease Trajectory-aware Transformer for Interpretable Patient Representation Learning

arXiv: https://arxiv.org/abs/2603.10180 · HTML: https://mlsanity.com/paper/2603.10180/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [GRAM: Graph-based Attention Model for Healthcare Representation Learning](https://mlsanity.com/paper/1611.07012/) (2016)
- [Pre-training of Graph Augmented Transformers for Medication Recommendation](https://mlsanity.com/paper/1906.00346/) (2019)
- [You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks](https://mlsanity.com/paper/2106.13264/) (2021)
- [Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer](https://mlsanity.com/paper/1906.04716/) (2019)
- [CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks](https://mlsanity.com/paper/2111.08585/) (2021)
- [DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency](https://mlsanity.com/paper/2403.06197/) (2024)
- [StageNet: Stage-Aware Neural Networks for Health Risk Prediction](https://mlsanity.com/paper/2001.10054/) (2020)
- [DeepCare: A Deep Dynamic Memory Model for Predictive Medicine](https://mlsanity.com/paper/1602.00357/) (2016)
- [Heterogeneous Similarity Graph Neural Network on Electronic Health Records](https://mlsanity.com/paper/2101.06800/) (2021)
- [Unsupervised pre-training of graph transformers on patient population graphs](https://mlsanity.com/paper/2207.10603/) (2022)

## Built on by (most cited first, 1 of 1)

- [ATHENA: Knowledge-guided agentic neural architecture search for AutoFormer-based electronic health record modeling](https://mlsanity.com/paper/2608.21712/) (2026)

## Abstract (arXiv)

The growing adoption of electronic health record (EHR) systems has provided unprecedented opportunities for predictive modeling to guide clinical decision making. Structured EHRs contain longitudinal observations of patients across hospital visits, where each visit is represented by a set of medical codes. While sequence-based, graph-based, and graph-enhanced sequence approaches have been developed to capture rich code interactions over time or within the same visits, they often overlook the inherent heterogeneous roles of medical codes arising from distinct clinical characteristics and contexts. To this end, in this study we propose the Disease Trajectory-aware Transformer for EHR (DT-BEHRT), a graph-enhanced sequential architecture that disentangles disease trajectories by explicitly modeling diagnosis-centric interactions within organ systems and capturing asynchronous progression patterns. To further enhance the representation robustness, we design a tailored pre-training methodology that combines trajectory-level code masking with ontology-informed ancestor prediction, promoting semantic alignment across multiple modeling modules. Extensive experiments on multiple benchmark datasets demonstrate that DT-BEHRT achieves strong predictive performance and provides interpretable patient representations that align with clinicians' disease-centered reasoning. The source code is publicly accessible at https://github.com/GatorAIM/DT-BEHRT.git.