# Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection

arXiv: https://arxiv.org/abs/1905.13456 · HTML: https://mlsanity.com/paper/1905.13456/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2019.

## Built on by (most cited first, 24 of 24)

- [Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation](https://mlsanity.com/paper/2102.04525/) (2021)
- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative models](https://mlsanity.com/paper/2111.14923/) (2021)
- [GAN-Based Data Augmentation and Anonymization for Skin-Lesion Analysis: A Critical Review](https://mlsanity.com/paper/2104.10603/) (2021)
- [Free-form tumor synthesis in computed tomography images via richer generative adversarial network](https://mlsanity.com/paper/2104.09701/) (2021)
- [Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging](https://mlsanity.com/paper/2107.09543/) (2021)
- [Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review](https://mlsanity.com/paper/2110.12076/) (2021)
- [Applications of Generative Adversarial Networks in Neuroimaging and Clinical Neuroscience](https://mlsanity.com/paper/2206.07081/) (2022)
- [Fruit Quality and Defect Image Classification with Conditional GAN Data Augmentation](https://mlsanity.com/paper/2104.05647/) (2021)
- [Breast Cancer Immunohistochemical Image Generation: a Benchmark Dataset and Challenge Review](https://mlsanity.com/paper/2305.03546/) (2023)
- [Generative Adversarial Networks for Data Augmentation](https://mlsanity.com/paper/2306.02019/) (2023)
- [DynaSeg: A Deep Dynamic Fusion Method for Unsupervised Image Segmentation Incorporating Feature Similarity and Spatial Continuity](https://mlsanity.com/paper/2405.05477/) (2024)
- [Enhanced Magnetic Resonance Image Synthesis with Contrast-Aware Generative Adversarial Networks](https://mlsanity.com/paper/2102.09386/) (2021)
- [Where is the disease? Semi-supervised pseudo-normality synthesis from an abnormal image](https://mlsanity.com/paper/2106.15345/) (2021)
- [Evaluating the Quality and Diversity of DCGAN-based Generatively Synthesized Diabetic Retinopathy Imagery](https://mlsanity.com/paper/2208.05593/) (2022)
- [Supervised Segmentation with Domain Adaptation for Small Sampled Orbital CT Images](https://mlsanity.com/paper/2107.00418/) (2021)
- [Deep Superpixel Generation and Clustering for Weakly Supervised Segmentation of Brain Tumors in MR Images](https://mlsanity.com/paper/2209.09930/) (2022)
- [BAPGAN: GAN-based Bone Age Progression of Femur and Phalange X-ray Images](https://mlsanity.com/paper/2110.08509/) (2021)
- [Generative Adversarial Networks for Weakly Supervised Generation and Evaluation of Brain Tumor Segmentations on MR Images](https://mlsanity.com/paper/2211.05269/) (2022)
- [Disentangled Uncertainty and Out of Distribution Detection in Medical Generative Models](https://mlsanity.com/paper/2211.06250/) (2022)
- [HSMix: Hard and Soft Mixing Data Augmentation for Medical Image Segmentation](https://mlsanity.com/paper/2511.17614/) (2025)
- [Improving Cardiac Risk Prediction Using Data Generation Techniques](https://mlsanity.com/paper/2512.20669/) (2025)
- [Diffusion-Based Tumor Inpainting for Renal Segmentation under Clinical Data Scarcity](https://mlsanity.com/paper/2609.21698/) (2026)
- [BrainAid: Efficient Brain Tumor Detection Using Lightweight CNNs and Explainable AI for Resource-Constrained Environments](https://mlsanity.com/paper/2610.15077/) (2026)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) achieve excellent computer-assisted diagnosis with sufficient annotated training data. However, most medical imaging datasets are small and fragmented. In this context, Generative Adversarial Networks (GANs) can synthesize realistic/diverse additional training images to fill the data lack in the real image distribution; researchers have improved classification by augmenting data with noise-to-image (e.g., random noise samples to diverse pathological images) or image-to-image GANs (e.g., a benign image to a malignant one). Yet, no research has reported results combining noise-to-image and image-to-image GANs for further performance boost. Therefore, to maximize the DA effect with the GAN combinations, we propose a two-step GAN-based DA that generates and refines brain Magnetic Resonance (MR) images with/without tumors separately: (i) Progressive Growing of GANs (PGGANs), multi-stage noise-to-image GAN for high-resolution MR image generation, first generates realistic/diverse 256 X 256 images; (ii) Multimodal UNsupervised Image-to-image Translation (MUNIT) that combines GANs/Variational AutoEncoders or SimGAN that uses a DA-focused GAN loss, further refines the texture/shape of the PGGAN-generated images similarly to the real ones. We thoroughly investigate CNN-based tumor classification results, also considering the influence of pre-training on ImageNet and discarding weird-looking GAN-generated images. The results show that, when combined with classic DA, our two-step GAN-based DA can significantly outperform the classic DA alone, in tumor detection (i.e., boosting sensitivity 93.67% to 97.48%) and also in other medical imaging tasks.