# Image-to-Image Translation Based on Deep Generative Modeling for Radiotherapy Synthetic Dataset Creation

arXiv: https://arxiv.org/abs/2410.01828 · HTML: https://mlsanity.com/paper/2410.01828/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training](https://mlsanity.com/paper/1703.10155/) (2017)
- [Generative Adversarial Networks for Image-to-Image Translation on Multi-Contrast MR Images - A Comparison of CycleGAN and UNIT](https://mlsanity.com/paper/1806.07777/) (2018)
- [Bridging the gap between paired and unpaired medical image translation](https://mlsanity.com/paper/2110.08407/) (2021)

## Abstract (arXiv)

Objective: Radiotherapy uses precise doses of radiation to treat cancer, requiring accurate verification, e.g. using the Electronic Portal Imaging Device (EPID), to guide treatment. To develop an effective artificial intelligence (AI) model for error detection and treatment verification, a large and well-annotated dataset of EPID images is needed, however, acquiring such high quality real data is difficult. While synthetic EPID data could be a viable alternative, it is critical to ensure that this data is as realistic as possible to effectively train an accurate and reliable AI model. The measurement uncertainty that is not modeled in EPID predictions but is present on real measured EPID images can hinder downstream tasks such as error detection and classification. Our research aims to improve synthetic EPID data through image-to-image (I2I) translation based on deep generative modeling. Approach: A dataset of 989 predicted EPID images and corresponding measured EPID images was used. We evaluate both paired and unpaired generative modeling approaches for this task. For the former, we introduce a novel modification of Variational Autoencoder (VAE) to I2I, a method that, to the best of our knowledge, has not been previously explored for this task. For the latter, we use UNsupervised Image-to-Image Translation Networks (UNIT). Results: Our results show that both models achieved some degree of I2I translation, with our novel modification of the VAE model outperforming the UNIT model in improving key metrics (mean absolute error: 4.1 cGy vs 6.4 cGy; relative dose difference in-field: 2.5% vs 5.5%; absolute dose difference in-field: 5.3 cGy vs 10.8 cGy). Significance: This enhanced synthetic data is expected to improve downstream tasks such as training neural networks for automated error detection and error classification in radiotherapy.