# Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling

arXiv: https://arxiv.org/abs/2409.09831 · HTML: https://mlsanity.com/paper/2409.09831/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 15 Sep 2024.

## Builds on (most cited first, 11 of 11)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020](https://mlsanity.com/paper/2104.10201/) (2021)
- [Can GPT-3.5 Generate and Code Discharge Summaries?](https://mlsanity.com/paper/2401.13512/) (2024)
- [Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts](https://mlsanity.com/paper/2309.13202/) (2023)
- [Exploration of Masked and Causal Language Modelling for Text Generation](https://mlsanity.com/paper/2405.12630/) (2024)
- [Textual Data Augmentation for Patient Outcomes Prediction](https://mlsanity.com/paper/2211.06778/) (2022)
- [Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modeling](https://mlsanity.com/paper/2401.01830/) (2024)
- [Exploring the Value of Pre-trained Language Models for Clinical Named Entity Recognition](https://mlsanity.com/paper/2210.12770/) (2022)
- [Generating Medical Prescriptions with Conditional Transformer](https://mlsanity.com/paper/2310.19727/) (2023)
- [Synthetic4Health: Generating Annotated Synthetic Clinical Letters](https://mlsanity.com/paper/2409.09501/) (2024)

## Built on by (most cited first, 2 of 2)

- [Synthetic Data for Veterinary EHR De-identification: Benefits, Limits, and Safety Trade-offs Under Fixed Compute](https://mlsanity.com/paper/2601.09756/) (2026)
- [The Safeguard Worked. Is the LLM System Safer?](https://mlsanity.com/paper/2609.00519/) (2026)

## Abstract (arXiv)

The vast amount of available medical records has the potential to improve healthcare and biomedical research. However, privacy restrictions make these data accessible for internal use only. Recent works have addressed this problem by generating synthetic data using Causal Language Modeling. Unfortunately, by taking this approach, it is often impossible to guarantee patient privacy while offering the ability to control the diversity of generations without increasing the cost of generating such data. In contrast, we present a system for generating synthetic free-text medical records using Masked Language Modeling. The system preserves critical medical information while introducing diversity in the generations and minimising re-identification risk. The system's size is about 120M parameters, minimising inference cost. The results demonstrate high-quality synthetic data with a HIPAA-compliant PHI recall rate of 96% and a re-identification risk of 3.5%. Moreover, downstream evaluations show that the generated data can effectively train a model with performance comparable to real data.