# Learning Foreground-Background Segmentation from Improved Layered GANs

arXiv: https://arxiv.org/abs/2104.00483 · HTML: https://mlsanity.com/paper/2104.00483/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 64 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [Large Scale Adversarial Representation Learning](https://mlsanity.com/paper/1907.02544/) (2019)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition](https://mlsanity.com/paper/2001.02407/) (2020)
- [BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images](https://mlsanity.com/paper/2002.08988/) (2020)
- [Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis](https://mlsanity.com/paper/1912.05237/) (2019)
- [Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering](https://mlsanity.com/paper/2010.09125/) (2020)
- [Controlling generative models with continuous factors of variations](https://mlsanity.com/paper/2001.10238/) (2020)
- [Unsupervised Object Segmentation by Redrawing](https://mlsanity.com/paper/1905.13539/) (2019)
- [Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge](https://mlsanity.com/paper/1609.09475/) (2016)
- [Unsupervised Learning of Image Segmentation Based on Differentiable Feature Clustering](https://mlsanity.com/paper/2007.09990/) (2020)

## Built on by (most cited first, 7 of 7)

- [FurryGAN: High Quality Foreground-aware Image Synthesis](https://mlsanity.com/paper/2208.10422/) (2022)
- [Foreground-Background Separation through Concept Distillation from Generative Image Foundation Models](https://mlsanity.com/paper/2212.14306/) (2022)
- [Self-Supervised 3D Human Pose Estimation in Static Video Via Neural Rendering](https://mlsanity.com/paper/2210.04514/) (2022)
- [ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentation](https://mlsanity.com/paper/2211.13974/) (2022)
- [Self-Supervised One-Shot Learning for Automatic Segmentation of StyleGAN Images](https://mlsanity.com/paper/2303.05639/) (2023)
- [LayeredDoc: Domain Adaptive Document Restoration with a Layer Separation Approach](https://mlsanity.com/paper/2406.08610/) (2024)
- [Weakly Supervised Object Segmentation by Background Conditional Divergence](https://mlsanity.com/paper/2506.22505/) (2025)

## Abstract (arXiv)

Deep learning approaches heavily rely on high-quality human supervision which is nonetheless expensive, time-consuming, and error-prone, especially for image segmentation task. In this paper, we propose a method to automatically synthesize paired photo-realistic images and segmentation masks for the use of training a foreground-background segmentation network. In particular, we learn a generative adversarial network that decomposes an image into foreground and background layers, and avoid trivial decompositions by maximizing mutual information between generated images and latent variables. The improved layered GANs can synthesize higher quality datasets from which segmentation networks of higher performance can be learned. Moreover, the segmentation networks are employed to stabilize the training of layered GANs in return, which are further alternately trained with Layered GANs. Experiments on a variety of single-object datasets show that our method achieves competitive generation quality and segmentation performance compared to related methods.