mlsanityData to 14 Oct 2026

Semantic-shape Adaptive Feature Modulation for Semantic Image Synthesis

arXiv 2203.16898 ↗First posted 31 Mar 2022cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed substantial progress in semantic image synthesis, it is still challenging in synthesizing photo-realistic images with rich details. Most previous methods focus on exploiting the given semantic map, which just captures an object-level layout for an image. Obviously, a fine-grained part-level semantic layout will benefit object details generation, and it can be roughly inferred from an object's shape. In order to exploit the part-level layouts, we propose a Shape-aware Position Descriptor (SPD) to describe each pixel's positional feature, where object shape is explicitly encoded into the SPD feature. Furthermore, a Semantic-shape Adaptive Feature Modulation (SAFM) block is proposed to combine the given semantic map and our positional features to produce adaptively modulated features. Extensive experiments demonstrate that the proposed SPD and SAFM significantly improve the generation of objects with rich details. Moreover, our method performs favorably against the SOTA methods in terms of quantitative and qualitative evaluation. The source code and model are available at https://github.com/cszy98/SAFM.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 31 Mar 2022.

Builds on

32 of 43 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  9. 2014Conditional Generative Adversarial Netscited by 2,162
  10. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  11. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
20 more
  1. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  2. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  3. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  4. 2018Self-Attention Generative Adversarial Networkscited by 809
  5. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  6. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 565
  7. 2017Dilated Residual Networkscited by 294
  8. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  9. 2017Geometric GANcited by 233
  10. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  11. 2019Controllable Text-to-Image Generationcited by 187
  12. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  13. 2020You Only Need Adversarial Supervision for Semantic Image Synthesiscited by 129
  14. 2019Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesiscited by 108
  15. 2016Semi-Supervised Learning with Generative Adversarial Networkscited by 103
  16. 2019Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generationcited by 75
  17. 2018Semi-parametric Image Synthesiscited by 50
  18. 2021Image Synthesis via Semantic Compositioncited by 45
  19. 2020SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objectscited by 43
  20. 2020Panoptic-based Image Synthesiscited by 23

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 5; 2025: 4.

16 papers.

Sort:

By area:

  1. 2022Semantic Image Synthesis via Diffusion Modelscited by 132
  2. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  3. 2023Freestyle Layout-to-Image Synthesiscited by 67
  4. 2022SceneComposer: Any-Level Semantic Image Synthesiscited by 43
  5. 2024PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesiscited by 20
  6. 2024SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flowcited by 20
  7. 2024Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesiscited by 11
  8. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  9. 2023Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesiscited by 8
  10. 2025Symmetrical Flow Matching: Unified Image Generation, Segmentation, and Classification with Score-Based Generative Modelscited by 8
  11. 2023Enhancing Object Coherence in Layout-to-Image Synthesiscited by 7
  12. 2024SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesiscited by 7
4 more
  1. 2023Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesiscited by 2
  2. 2023Variation-Aware Semantic Image Synthesiscited by 0
  3. 2024Reconciling Semantic Controllability and Diversity for Remote Sensing Image Synthesis with Hybrid Semantic Embeddingcited by 0
  4. 2025AnatoMaskGAN: GNN-Driven Slice Feature Fusion and Noise Augmentation for Medical Semantic Image Synthesiscited by 0