mlsanityData to 7 Oct 2026

STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generation

arXiv 2503.12213 ↗First posted 15 Mar 2025cs.CVcited by 3

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In layout-to-image (L2I) synthesis, controlled complex scenes are generated from coarse information like bounding boxes. Such a task is exciting to many downstream applications because the input layouts offer strong guidance to the generation process while remaining easily reconfigurable by humans. In this paper, we proposed STyled LAYout Diffusion (STAY Diffusion), a diffusion-based model that produces photo-realistic images and provides fine-grained control of stylized objects in scenes. Our approach learns a global condition for each layout, and a self-supervised semantic map for weight modulation using a novel Edge-Aware Normalization (EA Norm). A new Styled-Mask Attention (SM Attention) is also introduced to cross-condition the global condition and image feature for capturing the objects' relationships. These measures provide consistent guidance through the model, enabling more accurate and controllable image generation. Extensive benchmarking demonstrates that our STAY Diffusion presents high-quality images while surpassing previous state-of-the-art methods in generation diversity, accuracy, and controllability.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 52 of its 58 references that are in the map. First posted on arXiv 15 Mar 2025.

Builds on

52 of 58 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar
  2. 2023Freestyle Layout-to-Image Synthesis25 Mar
  3. 2023LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation16 Feb
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  7. 2022Pretraining is All You Need for Image-to-Image Translation25 May
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  10. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
30 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  6. 2014Conditional Generative Adversarial Netscited by 2,152
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  8. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  9. 2018Group Normalizationcited by 1,514
  10. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  11. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  13. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  14. 2018Self-Attention Generative Adversarial Networkscited by 809
  15. 2016Generative Adversarial Text to Image Synthesiscited by 735
  16. 2021Score-based Generative Modeling in Latent Spacecited by 634
  17. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  18. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  19. 2018cGANs with Projection Discriminatorcited by 253
  20. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 208
  21. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  22. 2022Semantic Image Synthesis via Diffusion Modelscited by 131
  23. 2020You Only Need Adversarial Supervision for Semantic Image Synthesiscited by 129
  24. 2018Image Generation from Layoutcited by 105
  25. 2019Image Synthesis From Reconfigurable Layout and Stylecited by 85
  26. 2019Specifying Object Attributes and Relations in Interactive Scene Generationcited by 76
  27. 2022Modeling Image Composition for Complex Scene Generationcited by 43
  28. 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
  29. 2021High-Resolution Complex Scene Synthesis with Transformerscited by 31
  30. 2022Interactive Image Synthesis with Panoptic Layout Generationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Unbiased Object Detection Beyond Frequency with Visually Prompted Image Synthesiscited by 4
  2. 2025Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Mapcited by 2
  3. 2026RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
  2. 2025A Two-Stage System for Layout-Controlled Image Generation using Large Language Models and Diffusion Models
  3. 2023Design Booster: A Text-Guided Diffusion Model for Image Translation with Spatial Layout Preservation
  4. 2025Control and Realism: Best of Both Worlds in Layout-to-Image without Training
  5. 2024Training-free Composite Scene Generation for Layout-to-Image Synthesis
  6. 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation
  7. 2023SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation
  8. 2024Enhancing Image Layout Control with Loss-Guided Diffusion Models