# Image Synthesis via Semantic Composition

arXiv: https://arxiv.org/abs/2109.07053 · HTML: https://mlsanity.com/paper/2109.07053/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 52 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Dynamic Filter Networks](https://mlsanity.com/paper/1605.09673/) (2016)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [CondConv: Conditionally Parameterized Convolutions for Efficient Inference](https://mlsanity.com/paper/1904.04971/) (2019)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Meta-SR: A Magnification-Arbitrary Network for Super-Resolution](https://mlsanity.com/paper/1903.00875/) (2019)
- [You Only Need Adversarial Supervision for Semantic Image Synthesis](https://mlsanity.com/paper/2012.04781/) (2020)
- [Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis](https://mlsanity.com/paper/1910.06809/) (2019)
- [Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation](https://mlsanity.com/paper/1904.06807/) (2019)

## Built on by (most cited first, 30 of 45)

- [Flow Matching in Latent Space](https://mlsanity.com/paper/2307.08698/) (2023)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [Freestyle Layout-to-Image Synthesis](https://mlsanity.com/paper/2303.14412/) (2023)
- [SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing](https://mlsanity.com/paper/2112.02236/) (2021)
- [SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation](https://mlsanity.com/paper/2308.10156/) (2023)
- [UniGS: Unified Representation for Image Generation and Segmentation](https://mlsanity.com/paper/2312.01985/) (2023)
- [DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation](https://mlsanity.com/paper/2207.06124/) (2022)
- [PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis](https://mlsanity.com/paper/2403.01852/) (2024)
- [SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow](https://mlsanity.com/paper/2405.20282/) (2024)
- [Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive](https://mlsanity.com/paper/2401.08815/) (2024)
- [Semantic-shape Adaptive Feature Modulation for Semantic Image Synthesis](https://mlsanity.com/paper/2203.16898/) (2022)
- [Retrieval-based Spatially Adaptive Normalization for Semantic Image Synthesis](https://mlsanity.com/paper/2204.02854/) (2022)
- [PalGAN: Image Colorization with Palette Generative Adversarial Networks](https://mlsanity.com/paper/2210.11204/) (2022)
- [DiffuMatting: Synthesizing Arbitrary Objects with Matting-level Annotation](https://mlsanity.com/paper/2403.06168/) (2024)
- [Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal](https://mlsanity.com/paper/2306.09008/) (2023)
- [Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis](https://mlsanity.com/paper/2402.16506/) (2024)
- [SCAM! Transferring humans between images with Semantic Cross Attention Modulation](https://mlsanity.com/paper/2210.04883/) (2022)
- [Text-guided Eyeglasses Manipulation with Spatial Constraints](https://mlsanity.com/paper/2304.12539/) (2023)
- [Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generation](https://mlsanity.com/paper/2306.00914/) (2023)
- [Learning 3D Photography Videos via Self-supervised Diffusion on Single Images](https://mlsanity.com/paper/2302.10781/) (2023)
- [Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis](https://mlsanity.com/paper/2307.12084/) (2023)
- [Symmetrical Flow Matching: Unified Image Generation, Segmentation, and Classification with Score-Based Generative Models](https://mlsanity.com/paper/2506.10634/) (2025)
- [Enhancing Object Coherence in Layout-to-Image Synthesis](https://mlsanity.com/paper/2311.10522/) (2023)
- [SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesis](https://mlsanity.com/paper/2403.09638/) (2024)
- [MM2Latent: Text-to-facial image generation and editing in GANs with multimodal assistance](https://mlsanity.com/paper/2409.11010/) (2024)
- [DRAN: Detailed Region-Adaptive Normalization for Conditional Image Synthesis](https://mlsanity.com/paper/2109.14525/) (2021)
- [PixelFolder: An Efficient Progressive Pixel Synthesis Network for Image Generation](https://mlsanity.com/paper/2204.00833/) (2022)
- [Controllable Image Generation via Collage Representations](https://mlsanity.com/paper/2304.13722/) (2023)
- [Automatic Generation of Semantic Parts for Face Image Synthesis](https://mlsanity.com/paper/2307.05317/) (2023)

## Abstract (arXiv)

In this paper, we present a novel approach to synthesize realistic images based on their semantic layouts. It hypothesizes that for objects with similar appearance, they share similar representation. Our method establishes dependencies between regions according to their appearance correlation, yielding both spatially variant and associated representations. Conditioning on these features, we propose a dynamic weighted network constructed by spatially conditional computation (with both convolution and normalization). More than preserving semantic distinctions, the given dynamic network strengthens semantic relevance, benefiting global structure and detail synthesis. We demonstrate that our method gives the compelling generation performance qualitatively and quantitatively with extensive experiments on benchmarks.