# Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer

arXiv: https://arxiv.org/abs/2502.20719 · HTML: https://mlsanity.com/paper/2502.20719/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Generating Multi-label Discrete Patient Records using Generative Adversarial Networks](https://mlsanity.com/paper/1703.06490/) (2017)
- [Zero Shot Health Trajectory Prediction Using Transformer](https://mlsanity.com/paper/2407.21124/) (2024)
- [Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applications](https://mlsanity.com/paper/2112.12047/) (2021)
- [CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Records](https://mlsanity.com/paper/2001.09346/) (2020)
- [EVA: Generating Longitudinal Electronic Health Records Using Conditional Variational Autoencoders](https://mlsanity.com/paper/2012.10020/) (2020)
- [Natural Language Generation for Electronic Health Records](https://mlsanity.com/paper/1806.01353/) (2018)
- [CEHR-GPT: Generating Electronic Health Records with Chronological Patient Timelines](https://mlsanity.com/paper/2402.04400/) (2024)
- [Generating Electronic Health Records with Multiple Data Types and Constraints](https://mlsanity.com/paper/2003.07904/) (2020)
- [The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithms](https://mlsanity.com/paper/2203.06369/) (2022)
- [Multimodal Medical Code Tokenizer](https://mlsanity.com/paper/2502.04397/) (2025)
- [A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sources](https://mlsanity.com/paper/2203.07018/) (2022)
- [CONAN: Complementary Pattern Augmentation for Rare Disease Detection](https://mlsanity.com/paper/1911.13232/) (2019)
- [Synthetic Data in Healthcare](https://mlsanity.com/paper/2304.03243/) (2023)
- [Collaborative Synthesis of Patient Records through Multi-Visit Health State Inference](https://mlsanity.com/paper/2312.14646/) (2023)

## Built on by (most cited first, 4 of 4)

- [Federated Timeline Synthesis: Scalable and Private Methodology For Model Training and Deployment](https://mlsanity.com/paper/2506.23358/) (2025)
- [From Statistical Fidelity to Clinical Consistency: Scalable Generation and Auditing of Synthetic Patient Trajectories](https://mlsanity.com/paper/2603.06720/) (2026)
- [HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds](https://mlsanity.com/paper/2605.10536/) (2026)
- [SynEHR: Joint Modeling Inter-visit Temporal Evolution and Intra-visit Clinical Structure for Longitudinal EHR Synthesis](https://mlsanity.com/paper/2608.21673/) (2026)

## Abstract (arXiv)

Generating realistic synthetic electronic health records (EHRs) holds tremendous promise for accelerating healthcare research, facilitating AI model development and enhancing patient privacy. However, existing generative methods typically treat EHRs as flat sequences of discrete medical codes. This approach overlooks two critical aspects: the inherent hierarchical organization of clinical coding systems and the rich semantic context provided by code descriptions. Consequently, synthetic patient sequences often lack high clinical fidelity and have limited utility in downstream clinical tasks. In this paper, we propose the Hierarchy- and Semantics-Guided Transformer (HiSGT), a novel framework that leverages both hierarchical and semantic information for the generative process. HiSGT constructs a hierarchical graph to encode parent-child and sibling relationships among clinical codes and employs a graph neural network to derive hierarchy-aware embeddings. These are then fused with semantic embeddings extracted from a pre-trained clinical language model (e.g., ClinicalBERT), enabling the Transformer-based generator to more accurately model the nuanced clinical patterns inherent in real EHRs. Extensive experiments on the MIMIC-III and MIMIC-IV datasets demonstrate that HiSGT significantly improves the statistical alignment of synthetic data with real patient records, as well as supports robust downstream applications such as chronic disease classification. By addressing the limitations of conventional raw code-based generative models, HiSGT represents a significant step toward clinically high-fidelity synthetic data generation and a general paradigm suitable for interpretable medical code representation, offering valuable applications in data augmentation and privacy-preserving healthcare analytics.