mlsanityData to 7 Oct 2026

Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis

arXiv 1910.06809 ↗First posted 15 Oct 2019cs.CV cs.LG eess.IVcited by 108

Not on the map. Why?

Abstract (arXiv)

Semantic image synthesis aims at generating photorealistic images from semantic layouts. Previous approaches with conditional generative adversarial networks (GAN) show state-of-the-art performance on this task, which either feed the semantic label maps as inputs to the generator, or use them to modulate the activations in normalization layers via affine transformations. We argue that convolutional kernels in the generator should be aware of the distinct semantic labels at different locations when generating images. In order to better exploit the semantic layout for the image generator, we propose to predict convolutional kernels conditioned on the semantic label map to generate the intermediate feature maps from the noise maps and eventually generate the images. Moreover, we propose a feature pyramid semantics-embedding discriminator, which is more effective in enhancing fine details and semantic alignments between the generated images and the input semantic layouts than previous multi-scale discriminators. We achieve state-of-the-art results on both quantitative metrics and subjective evaluation on various semantic segmentation datasets, demonstrating the effectiveness of our approach.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 35; 2022: 21; 2023: 33; 2024: 13; 2025: 6.

108 papers.

Sort:

By area:

  1. 2021Dynamic Neural Networks: A Surveycited by 405
  2. 2022BBDM: Image-to-image Translation with Brownian Bridge Diffusion Modelscited by 222
  3. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  4. 2022Dress Code: High-Resolution Multi-Category Virtual Try-Oncited by 161
  5. 2022Semantic Image Synthesis via Diffusion Modelscited by 131
  6. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  7. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  8. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  9. 2021Enhancing Photorealism Enhancementcited by 74
  10. 2023Freestyle Layout-to-Image Synthesiscited by 67
  11. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  12. 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
38 more
  1. 2021Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Datacited by 53
  2. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  3. 2021Diverse Semantic Image Synthesis via Probability Distribution Modelingcited by 49
  4. 2021StyleAlign: Analysis and Applications of Aligned StyleGAN Modelscited by 48
  5. 2021Image Synthesis via Semantic Compositioncited by 45
  6. 2021Learning Semantic Person Image Generation by Region-Adaptive Normalizationcited by 44
  7. 2022Modeling Image Composition for Complex Scene Generationcited by 43
  8. 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
  9. 2023Generative Semantic Communication: Diffusion Models Beyond Bit Recoverycited by 41
  10. 2022MobileFaceSwap: A Lightweight Framework for Video Face Swappingcited by 39
  11. 2022High-Fidelity Guided Image Synthesis with Latent Diffusion Modelscited by 36
  12. 2021Learned Spatial Representations for Few-shot Talking-Head Synthesiscited by 32
  13. 2021Dual Contrastive Loss and Attention for GANscited by 30
  14. 2023SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generationcited by 29
  15. 2022Cross-View Panorama Image Synthesiscited by 26
  16. 2021Collaging Class-specific GANs for Semantic Image Synthesiscited by 25
  17. 2021Compositional Transformers for Scene Generationcited by 24
  18. 2023Generated Distributions Are All You Need for Membership Inference Attacks Against Generative Modelscited by 22
  19. 2021Long-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Datacited by 21
  20. 2023Video ControlNet: Towards Temporally Consistent Synthetic-to-Real Video Translation Using Conditional Image Diffusion Modelscited by 20
  21. 2024PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesiscited by 20
  22. 2024SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flowcited by 20
  23. 2023SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Controlcited by 18
  24. 2022Semantic-shape Adaptive Feature Modulation for Semantic Image Synthesiscited by 16
  25. 2022Retrieval-based Spatially Adaptive Normalization for Semantic Image Synthesiscited by 15
  26. 2022PalGAN: Image Colorization with Palette Generative Adversarial Networkscited by 15
  27. 2021Semantic Palette: Guiding Scene Generation with Class Proportionscited by 14
  28. 2023Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removalcited by 12
  29. 2023Synthesis-based Imaging-Differentiation Representation Learning for Multi-Sequence 3D/4D MRIcited by 11
  30. 2022SCAM! Transferring humans between images with Semantic Cross Attention Modulationcited by 10
  31. 2024Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesiscited by 10
  32. 2023DejaVu: Conditional Regenerative Learning to Enhance Dense Predictioncited by 9
  33. 2023Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generationcited by 9
  34. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  35. 2021Deep Image Synthesis from Intuitive User Input: A Review and Perspectivescited by 8
  36. 2021Layout-to-Image Translation with Double Pooling Generative Adversarial Networkscited by 8
  37. 2021USIS: Unsupervised Semantic Image Synthesiscited by 8
  38. 2022Local and Global GANs with Semantic-Aware Upsampling for Image Generationcited by 8

50 most cited of 108 All 108 (JSON)