# The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithms

arXiv: https://arxiv.org/abs/2203.06369 · HTML: https://mlsanity.com/paper/2203.06369/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 105 references that are in the map.
- First posted on arXiv 12 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [Generating Multi-label Discrete Patient Records using Generative Adversarial Networks](https://mlsanity.com/paper/1703.06490/) (2017)
- [DeepMind Lab](https://mlsanity.com/paper/1612.03801/) (2016)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Adversarial Feature Matching for Text Generation](https://mlsanity.com/paper/1706.03850/) (2017)
- [Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applications](https://mlsanity.com/paper/2112.12047/) (2021)
- [Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions](https://mlsanity.com/paper/2002.03478/) (2020)
- [Airline Passenger Name Record Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1807.06657/) (2018)
- [Frustratingly Short Attention Spans in Neural Language Modeling](https://mlsanity.com/paper/1702.04521/) (2017)
- [Language Generation with Recurrent Generative Adversarial Networks without Pre-training](https://mlsanity.com/paper/1706.01399/) (2017)
- [Regional Tree Regularization for Interpretability in Black Box Models](https://mlsanity.com/paper/1908.04494/) (2019)
- [Offline reinforcement learning with uncertainty for treatment strategies in sepsis](https://mlsanity.com/paper/2107.04491/) (2021)
- [Synthetic Acute Hypotension and Sepsis Datasets Based on MIMIC-III and Published as Part of the Health Gym Project](https://mlsanity.com/paper/2112.03914/) (2021)

## Built on by (most cited first, 19 of 19)

- [Deep Imputation of Missing Values in Time Series Health Data: A Review with Benchmarking](https://mlsanity.com/paper/2302.10902/) (2023)
- [Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare](https://mlsanity.com/paper/2401.13716/) (2024)
- [Generating Synthetic Clinical Data that Capture Class Imbalanced Distributions with Generative Adversarial Networks: Example using Antiretroviral Therapy for HIV](https://mlsanity.com/paper/2208.08655/) (2022)
- [D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning](https://mlsanity.com/paper/2408.08441/) (2024)
- [Synthetic Health-related Longitudinal Data with Mixed-type Variables Generated using Diffusion Models](https://mlsanity.com/paper/2303.12281/) (2023)
- [Transfer Learning of Tabular Data by Finetuning Large Language Models](https://mlsanity.com/paper/2501.06863/) (2025)
- [Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer](https://mlsanity.com/paper/2502.20719/) (2025)
- [Mitigating Health Data Poverty: Generative Approaches versus Resampling for Time-series Clinical Data](https://mlsanity.com/paper/2210.13958/) (2022)
- [Masked Clinical Modelling: A Framework for Synthetic and Augmented Survival Data Generation](https://mlsanity.com/paper/2410.16811/) (2024)
- [Methods for generating and evaluating synthetic longitudinal patient data: a systematic review](https://mlsanity.com/paper/2309.12380/) (2023)
- [Attention-Based Synthetic Data Generation for Calibration-Enhanced Survival Analysis: A Case Study for Chronic Kidney Disease Using Electronic Health Records](https://mlsanity.com/paper/2503.06096/) (2025)
- [PRIME-CVD: A Parametrically Rendered Informatics Medical Environment for Education in Cardiovascular Risk Modelling](https://mlsanity.com/paper/2603.19299/) (2026)
- [Joint Distributional Learning via Cramer-Wold Distance](https://mlsanity.com/paper/2310.16374/) (2023)
- [Balanced Marginal and Joint Distributional Learning via Mixture Cramer-Wold Distance](https://mlsanity.com/paper/2312.03307/) (2023)
- [Critical Challenges and Guidelines in Evaluating Synthetic Tabular Data: A Systematic Review](https://mlsanity.com/paper/2504.18544/) (2025)
- [Evaluation metrics for temporal preservation in synthetic longitudinal patient data](https://mlsanity.com/paper/2602.10643/) (2026)
- [Counterfactually Safe Reinforcement Learning](https://mlsanity.com/paper/2605.25114/) (2026)
- [Synthetic but Not Realistic: The Evaluation Challenge in Generative Modelling for Structured Electronic Medical Records](https://mlsanity.com/paper/2606.08903/) (2026)
- [Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learning](https://mlsanity.com/paper/2606.19481/) (2026)

## Abstract (arXiv)

In recent years, the machine learning research community has benefited tremendously from the availability of openly accessible benchmark datasets. Clinical data are usually not openly available due to their highly confidential nature. This has hampered the development of reproducible and generalisable machine learning applications in health care. Here we introduce the Health Gym - a growing collection of highly realistic synthetic medical datasets that can be freely accessed to prototype, evaluate, and compare machine learning algorithms, with a specific focus on reinforcement learning. The three synthetic datasets described in this paper present patient cohorts with acute hypotension and sepsis in the intensive care unit, and people with human immunodeficiency virus (HIV) receiving antiretroviral therapy in ambulatory care. The datasets were created using a novel generative adversarial network (GAN). The distributions of variables, and correlations between variables and trends over time in the synthetic datasets mirror those in the real datasets. Furthermore, the risk of sensitive information disclosure associated with the public distribution of the synthetic datasets is estimated to be very low.