# Controllable Image Synthesis via SegVAE

arXiv: https://arxiv.org/abs/2007.08397 · HTML: https://mlsanity.com/paper/2007.08397/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2020.

## Built on by (most cited first, 13 of 13)

- [In&Out : Diverse Image Outpainting via GAN Inversion](https://mlsanity.com/paper/2104.00675/) (2021)
- [Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Model](https://mlsanity.com/paper/2208.12675/) (2022)
- [Guided Co-Modulated GAN for 360deg Field of View Extrapolation](https://mlsanity.com/paper/2204.07286/) (2022)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [Vector Quantized Image-to-Image Translation](https://mlsanity.com/paper/2207.13286/) (2022)
- [Action2video: Generating Videos of Human 3D Actions](https://mlsanity.com/paper/2111.06925/) (2021)
- [The Image Local Autoregressive Transformer](https://mlsanity.com/paper/2106.02514/) (2021)
- [LatentKeypointGAN: Controlling Images via Latent Keypoints](https://mlsanity.com/paper/2103.15812/) (2021)
- [MixSyn: Learning Composition and Style for Multi-Source Image Synthesis](https://mlsanity.com/paper/2111.12705/) (2021)
- [Range-Edit: Semantic Mask Guided Outdoor LiDAR Scene Editing](https://mlsanity.com/paper/2511.17269/) (2025)
- [MeronymNet: A Hierarchical Approach for Unified and Controllable Multi-Category Object Generation](https://mlsanity.com/paper/2110.08818/) (2021)
- [LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract](https://mlsanity.com/paper/2205.03448/) (2022)
- [SketchDeco: Training-Free Latent Composition for Precise Sketch Colourisation](https://mlsanity.com/paper/2405.18716/) (2024)

## Abstract (arXiv)

Flexible user controls are desirable for content creation and image editing. A semantic map is commonly used intermediate representation for conditional image generation. Compared to the operation on raw RGB pixels, the semantic map enables simpler user modification. In this work, we specifically target at generating semantic maps given a label-set consisting of desired categories. The proposed framework, SegVAE, synthesizes semantic maps in an iterative manner using conditional variational autoencoder. Quantitative and qualitative experiments demonstrate that the proposed model can generate realistic and diverse semantic maps. We also apply an off-the-shelf image-to-image translation model to generate realistic RGB images to better understand the quality of the synthesized semantic maps. Furthermore, we showcase several real-world image-editing applications including object removal, object insertion, and object replacement.