# Semantic Image Synthesis via Class-Adaptive Cross-Attention

arXiv: https://arxiv.org/abs/2308.16071 · HTML: https://mlsanity.com/paper/2308.16071/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 40 references that are in the map.
- First posted on arXiv 30 Aug 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [EditGAN: High-Precision Semantic Image Editing](https://mlsanity.com/paper/2111.03186/) (2021)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis](https://mlsanity.com/paper/1910.06809/) (2019)
- [Semantically Multi-modal Image Synthesis](https://mlsanity.com/paper/2003.12697/) (2020)
- [Efficient Semantic Image Synthesis via Class-Adaptive Normalization](https://mlsanity.com/paper/2012.04644/) (2020)
- [SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing](https://mlsanity.com/paper/2112.02236/) (2021)
- [Diverse Semantic Image Synthesis via Probability Distribution Modeling](https://mlsanity.com/paper/2103.06878/) (2021)
- [Image Synthesis via Semantic Composition](https://mlsanity.com/paper/2109.07053/) (2021)
- [Collaging Class-specific GANs for Semantic Image Synthesis](https://mlsanity.com/paper/2110.04281/) (2021)

## Built on by (most cited first, 5 of 5)

- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [Enriching Information and Preserving Semantic Consistency in Expanding Curvilinear Object Segmentation Datasets](https://mlsanity.com/paper/2407.08209/) (2024)
- [Adversarial Identity Injection for Semantic Face Image Synthesis](https://mlsanity.com/paper/2404.10408/) (2024)
- [SISMA: Semantic Face Image Synthesis with Mamba](https://mlsanity.com/paper/2509.17651/) (2025)
- [CrackForward: Context-Aware Severity Stage Crack Synthesis for Data Augmentation](https://mlsanity.com/paper/2604.19941/) (2026)

## Abstract (arXiv)

In semantic image synthesis the state of the art is dominated by methods that use customized variants of the SPatially-Adaptive DE-normalization (SPADE) layers, which allow for good visual generation quality and editing versatility. By design, such layers learn pixel-wise modulation parameters to de-normalize the generator activations based on the semantic class each pixel belongs to. Thus, they tend to overlook global image statistics, ultimately leading to unconvincing local style editing and causing global inconsistencies such as color or illumination distribution shifts. Also, SPADE layers require the semantic segmentation mask for mapping styles in the generator, preventing shape manipulations without manual intervention. In response, we designed a novel architecture where cross-attention layers are used in place of SPADE for learning shape-style correlations and so conditioning the image generation process. Our model inherits the versatility of SPADE, at the same time obtaining state-of-the-art generation quality, as well as improved global and local style transfer. Code and models available at https://github.com/TFonta/CA2SIS.