# Adversarial Synthesis Learning Enables Segmentation Without Target Modality Ground Truth

arXiv: https://arxiv.org/abs/1712.07695 · HTML: https://mlsanity.com/paper/1712.07695/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2017.

## Built on by (most cited first, 30 of 30)

- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [Source-Free Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2108.03152/) (2021)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization](https://mlsanity.com/paper/2102.06315/) (2021)
- [Unsupervised Model Adaptation for Source-free Segmentation of Medical Images](https://mlsanity.com/paper/2211.00807/) (2022)
- [Domain Adaptation for the Segmentation of Confidential Medical Images](https://mlsanity.com/paper/2101.00522/) (2021)
- [Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration](https://mlsanity.com/paper/2104.07056/) (2021)
- [Generalize Ultrasound Image Segmentation via Instant and Plug & Play Style Transfer](https://mlsanity.com/paper/2101.03711/) (2021)
- [Unsupervised Cross-Modality Domain Adaptation for Segmenting Vestibular Schwannoma and Cochlea with Data Augmentation and Model Ensemble](https://mlsanity.com/paper/2109.12169/) (2021)
- [CS2: A Controllable and Simultaneous Synthesizer of Images and Annotations with Minimal Human Intervention](https://mlsanity.com/paper/2206.13394/) (2022)
- [Medical Image Segmentation with Domain Adaptation: A Survey](https://mlsanity.com/paper/2311.01702/) (2023)
- [STRUDEL: Self-Training with Uncertainty Dependent Label Refinement across Domains](https://mlsanity.com/paper/2104.11596/) (2021)
- [Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images](https://mlsanity.com/paper/2107.14449/) (2021)
- [MS-MT: Multi-Scale Mean Teacher with Contrastive Unpaired Translation for Cross-Modality Vestibular Schwannoma and Cochlea Segmentation](https://mlsanity.com/paper/2303.15826/) (2023)
- [Learning Modality-Aware Representations: Adaptive Group-wise Interaction Network for Multimodal MRI Synthesis](https://mlsanity.com/paper/2411.14684/) (2024)
- [ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations](https://mlsanity.com/paper/2101.00567/) (2021)
- [Random Multi-Channel Image Synthesis for Multiplexed Immunofluorescence Imaging](https://mlsanity.com/paper/2109.09004/) (2021)
- [Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN](https://mlsanity.com/paper/2204.03082/) (2022)
- [Generative Adversarial Networks and Other Generative Models](https://mlsanity.com/paper/2207.03887/) (2022)
- [CNNs and GANs in MRI-based cross-modality medical image estimation](https://mlsanity.com/paper/2106.02198/) (2021)
- [Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) for Remote Land-use Change Detection](https://mlsanity.com/paper/2303.14322/) (2023)
- [Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning](https://mlsanity.com/paper/2403.02566/) (2024)
- [Realistic Endoscopic Image Generation Method Using Virtual-to-real Image-domain Translation](https://mlsanity.com/paper/2201.04918/) (2022)
- [ResNCT: A Deep Learning Model for the Synthesis of Nephrographic Phase Images in CT Urography](https://mlsanity.com/paper/2405.04629/) (2024)
- [Cross Modality Medical Image Synthesis for Improving Liver Segmentation](https://mlsanity.com/paper/2503.00945/) (2025)
- [Producing Histopathology Phantom Images using Generative Adversarial Networks to improve Tumor Detection](https://mlsanity.com/paper/2205.10691/) (2022)
- [Contrastive Image Synthesis and Self-supervised Feature Adaptation for Cross-Modality Biomedical Image Segmentation](https://mlsanity.com/paper/2207.13240/) (2022)
- [From fat droplets to floating forests: cross-domain transfer learning using a PatchGAN-based segmentation model](https://mlsanity.com/paper/2211.03937/) (2022)
- [Training a Computer Vision Model for Commercial Bakeries with Primarily Synthetic Images](https://mlsanity.com/paper/2409.20122/) (2024)
- [ORGAN: Object-Centric Representation Learning using Cycle Consistent Generative Adversarial Networks](https://mlsanity.com/paper/2603.02063/) (2026)

## Abstract (arXiv)

A lack of generalizability is one key limitation of deep learning based segmentation. Typically, one manually labels new training images when segmenting organs in different imaging modalities or segmenting abnormal organs from distinct disease cohorts. The manual efforts can be alleviated if one is able to reuse manual labels from one modality (e.g., MRI) to train a segmentation network for a new modality (e.g., CT). Previously, two stage methods have been proposed to use cycle generative adversarial networks (CycleGAN) to synthesize training images for a target modality. Then, these efforts trained a segmentation network independently using synthetic images. However, these two independent stages did not use the complementary information between synthesis and segmentation. Herein, we proposed a novel end-to-end synthesis and segmentation network (EssNet) to achieve the unpaired MRI to CT image synthesis and CT splenomegaly segmentation simultaneously without using manual labels on CT. The end-to-end EssNet achieved significantly higher median Dice similarity coefficient (0.9188) than the two stages strategy (0.8801), and even higher than canonical multi-atlas segmentation (0.9125) and ResNet method (0.9107), which used the CT manual labels.