# PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis

arXiv: https://arxiv.org/abs/2403.01852 · HTML: https://mlsanity.com/paper/2403.01852/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 53 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 30 of 48)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2305.16322/) (2023)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)
- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [SpaText: Spatio-Textual Representation for Controllable Image Generation](https://mlsanity.com/paper/2211.14305/) (2022)
- [Grounded Text-to-Image Synthesis with Attention Refocusing](https://mlsanity.com/paper/2306.05427/) (2023)
- [Dense Text-to-Image Generation with Attention Modulation](https://mlsanity.com/paper/2308.12964/) (2023)
- [Pretraining is All You Need for Image-to-Image Translation](https://mlsanity.com/paper/2205.12952/) (2022)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)

## Built on by (most cited first, 20 of 20)

- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [VideoElevator: Elevating Video Generation Quality with Versatile Text-to-Image Diffusion Models](https://mlsanity.com/paper/2403.05438/) (2024)
- [FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors](https://mlsanity.com/paper/2501.08225/) (2025)
- [Seg2Any: Open-set Segmentation-Mask-to-Image Generation with Precise Shape and Semantic Control](https://mlsanity.com/paper/2506.00596/) (2025)
- [InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attention](https://mlsanity.com/paper/2509.16691/) (2025)
- [SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesis](https://mlsanity.com/paper/2403.09638/) (2024)
- [coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation](https://mlsanity.com/paper/2603.12829/) (2026)
- [A Unified Image-Dense Annotation Generation Model for Underwater Scenes](https://mlsanity.com/paper/2503.21771/) (2025)
- [Improving Editability in Image Generation with Layer-wise Memory](https://mlsanity.com/paper/2505.01079/) (2025)
- [FICGen: Frequency-Inspired Contextual Disentanglement for Layout-driven Degraded Image Generation](https://mlsanity.com/paper/2509.01107/) (2025)
- [MasterWeaver: Taming Editability and Face Identity for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2405.05806/) (2024)
- [Boosting Few-Shot Detection with Large Language Models and Layout-to-Image Synthesis](https://mlsanity.com/paper/2410.06841/) (2024)
- [OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation](https://mlsanity.com/paper/2605.21343/) (2026)
- [To Blend In, First Decouple: Rethinking Camouflage Image Generation via Context-Decoupled Representations](https://mlsanity.com/paper/2607.17768/) (2026)
- [Reconciling Semantic Controllability and Diversity for Remote Sensing Image Synthesis with Hybrid Semantic Embedding](https://mlsanity.com/paper/2411.14781/) (2024)
- [StructDiff: Structure-aware Diffusion Model for 3D Fine-grained Medical Image Synthesis](https://mlsanity.com/paper/2503.09560/) (2025)
- [MALeR: Improving Compositional Fidelity in Layout-Guided Generation](https://mlsanity.com/paper/2511.06002/) (2025)
- [InstancePin: Instance-Addressable Layout-to-Image Diffusion via Coordinate Pinning](https://mlsanity.com/paper/2608.00588/) (2026)
- [Bridging Language and Spherical Space: Object-Centric Control for Text-to-Panorama Generation](https://mlsanity.com/paper/2608.20691/) (2026)

## Abstract (arXiv)

Recent advancements in large-scale pre-trained text-to-image models have led to remarkable progress in semantic image synthesis. Nevertheless, synthesizing high-quality images with consistent semantics and layout remains a challenge. In this paper, we propose the adaPtive LAyout-semantiC fusion modulE (PLACE) that harnesses pre-trained models to alleviate the aforementioned issues. Specifically, we first employ the layout control map to faithfully represent layouts in the feature space. Subsequently, we combine the layout and semantic features in a timestep-adaptive manner to synthesize images with realistic details. During fine-tuning, we propose the Semantic Alignment (SA) loss to further enhance layout alignment. Additionally, we introduce the Layout-Free Prior Preservation (LFP) loss, which leverages unlabeled data to maintain the priors of pre-trained models, thereby improving the visual quality and semantic consistency of synthesized images. Extensive experiments demonstrate that our approach performs favorably in terms of visual quality, semantic consistency, and layout alignment. The source code and model are available at https://github.com/cszy98/PLACE/tree/main.