# GANBERT: Generative Adversarial Networks with Bidirectional Encoder Representations from Transformers for MRI to PET synthesis

arXiv: https://arxiv.org/abs/2008.04393 · HTML: https://mlsanity.com/paper/2008.04393/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2020.

## Built on by (most cited first, 15 of 15)

- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT](https://mlsanity.com/paper/2304.04920/) (2023)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [Swin Deformable Attention U-Net Transformer (SDAUT) for Explainable Fast MRI](https://mlsanity.com/paper/2207.02390/) (2022)
- [PASTA: Pathology-Aware MRI to PET Cross-Modal Translation with Diffusion Models](https://mlsanity.com/paper/2405.16942/) (2024)
- [Generative Adversarial Networks for Brain Images Synthesis: A Review](https://mlsanity.com/paper/2305.15421/) (2023)
- [Plasma-CycleGAN: Plasma Biomarker-Guided MRI to PET Cross-modality Translation Using Conditional CycleGAN](https://mlsanity.com/paper/2501.02146/) (2025)
- [Transformer Networks for Data Augmentation of Human Physical Activity Recognition](https://mlsanity.com/paper/2109.01081/) (2021)
- [Brain MRI-to-PET Synthesis using 3D Convolutional Attention Networks](https://mlsanity.com/paper/2211.12082/) (2022)
- [Multi-task Deep Learning for Cerebrovascular Disease Classification and MRI-to-PET Translation](https://mlsanity.com/paper/2202.06142/) (2022)
- [Connectional-Style-Guided Contextual Representation Learning for Brain Disease Diagnosis](https://mlsanity.com/paper/2306.05297/) (2023)
- [MCR-VQGAN: A Scalable and Cost-Effective Tau PET Synthesis Approach for Alzheimer's Disease Imaging](https://mlsanity.com/paper/2512.15947/) (2025)
- [Synthesizing beta-amyloid PET images from T1-weighted Structural MRI: A Preliminary Study](https://mlsanity.com/paper/2409.18282/) (2024)
- [Pyramid Hierarchical Masked Diffusion Model for Imaging Synthesis](https://mlsanity.com/paper/2507.16579/) (2025)

## Abstract (arXiv)

Synthesizing medical images, such as PET, is a challenging task due to the fact that the intensity range is much wider and denser than those in photographs and digital renderings and are often heavily biased toward zero. Above all, intensity values in PET have absolute significance, and are used to compute parameters that are reproducible across the population. Yet, usually much manual adjustment has to be made in pre-/post- processing when synthesizing PET images, because its intensity ranges can vary a lot, e.g., between -100 to 1000 in floating point values. To overcome these challenges, we adopt the Bidirectional Encoder Representations from Transformers (BERT) algorithm that has had great success in natural language processing (NLP), where wide-range floating point intensity values are represented as integers ranging between 0 to 10000 that resemble a dictionary of natural language vocabularies. BERT is then trained to predict a proportion of masked values images, where its "next sentence prediction (NSP)" acts as GAN discriminator. Our proposed approach, is able to generate PET images from MRI images in wide intensity range, with no manual adjustments in pre-/post- processing. It is a method that can scale and ready to deploy.