# TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors

arXiv: https://arxiv.org/abs/2102.02690 · HTML: https://mlsanity.com/paper/2102.02690/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 61 references that are in the map.
- First posted on arXiv 4 Feb 2021.

## Builds on (most cited first, 30 of 34)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation](https://mlsanity.com/paper/1908.10454/) (2019)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation](https://mlsanity.com/paper/1810.07842/) (2018)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [Richer Convolutional Features for Edge Detection](https://mlsanity.com/paper/1612.02103/) (2016)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [A Review on Deep Learning Techniques Applied to Semantic Segmentation](https://mlsanity.com/paper/1704.06857/) (2017)
- [TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation](https://mlsanity.com/paper/1801.05746/) (2018)
- [Unsupervised Object Segmentation by Redrawing](https://mlsanity.com/paper/1905.13539/) (2019)
- [Semantic Segmentation using Adversarial Networks](https://mlsanity.com/paper/1611.08408/) (2016)
- [Efficient piecewise training of deep structured models for semantic segmentation](https://mlsanity.com/paper/1504.01013/) (2015)
- [Fast Edge Detection Using Structured Forests](https://mlsanity.com/paper/1406.5549/) (2014)
- [W-Net: A Deep Model for Fully Unsupervised Image Segmentation](https://mlsanity.com/paper/1711.08506/) (2017)
- [SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation](https://mlsanity.com/paper/1706.01805/) (2017)
- [Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields](https://mlsanity.com/paper/1610.02177/) (2016)
- [Synthetic Medical Images from Dual Generative Adversarial Networks](https://mlsanity.com/paper/1709.01872/) (2017)
- [Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities](https://mlsanity.com/paper/1704.03379/) (2017)
- [Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/1506.04924/) (2015)
- [Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks](https://mlsanity.com/paper/1706.09318/) (2017)
- [Automatic Liver Segmentation Using an Adversarial Image-to-Image Network](https://mlsanity.com/paper/1707.08037/) (2017)

## Abstract (arXiv)

Medical image segmentation is routinely performed to isolate regions of interest, such as organs and lesions. Currently, deep learning is the state of the art for automatic segmentation, but is usually limited by the need for supervised training with large datasets that have been manually segmented by trained clinicians. The goal of semi-superised and unsupervised image segmentation is to greatly reduce, or even eliminate, the need for training data and therefore to minimze the burden on clinicians when training segmentation models. To this end we introduce a novel network architecture for capable of unsupervised and semi-supervised image segmentation called TricycleGAN. This approach uses three generative models to learn translations between medical images and segmentation maps using edge maps as an intermediate step. Distinct from other approaches based on generative networks, TricycleGAN relies on shape priors rather than colour and texture priors. As such, it is particularly well-suited for several domains of medical imaging, such as ultrasound imaging, where commonly used visual cues may be absent. We present experiments with TricycleGAN on a clinical dataset of kidney ultrasound images and the benchmark ISIC 2018 skin lesion dataset.