mlsanityData to 14 Oct 2026

Controllable Coupled Image Generation via Diffusion Models

arXiv 2506.06826 ↗First posted 7 Jun 2025cs.CV cs.AIcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We provide an attention-level control method for the task of coupled image generation, where "coupled" means that multiple simultaneously generated images are expected to have the same or very similar backgrounds. While backgrounds coupled, the centered objects in the generated images are still expected to enjoy the flexibility raised from different text prompts. The proposed method disentangles the background and entity components in the model's cross-attention modules, attached with a sequence of time-varying weight control parameters depending on the time step of sampling. We optimize this sequence of weight control parameters with a combined objective that assesses how coupled the backgrounds are as well as text-to-image alignment and overall visual quality. Empirical results demonstrate that our method outperforms existing approaches across these criteria.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 55 of its 67 references that are in the map. First posted on arXiv 7 Jun 2025.

Builds on

55 of 67 references.

Closest ancestors · 23

Same direction, newest first

  1. 2024Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models11 Nov
  2. 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations14 Oct
  3. 2024DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition12 Jun
  4. 2024DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations11 Mar
  5. 2024Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation8 Mar
  6. 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervision6 Dec
  7. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editing27 Sep
  8. 2023Dense Text-to-Image Generation with Attention Modulation24 Aug
  9. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models15 Aug
  10. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  11. 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
  12. 2023LayerDiffusion: Layered Controlled Image Editing with Diffusion Models30 May
11 more
  1. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  2. 2023PARASOL: Parametric Style Control for Diffusion Image Synthesis11 Mar
  3. 2023Directed Diffusion: Direct Control of Object Placement through Attention Guidance25 Feb
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  6. 2022Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models16 Dec
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  8. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  11. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2023Segment Anythingcited by 8,234
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2022Generative Adversarial Networkscited by 5,627
  10. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
20 more
  1. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  2. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  3. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  4. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  5. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  6. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  7. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 327
  8. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  9. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  10. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  11. 2022Perception Prioritized Training of Diffusion Modelscited by 303
  12. 2016Semantic Image Inpainting with Deep Generative Modelscited by 157
  13. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometrycited by 125
  14. 2024Video Interpolation with Diffusion Modelscited by 68
  15. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scalecited by 49
  16. 2018Context-Aware Synthesis and Placement of Object Instancescited by 41
  17. 2024Exploring Diffusion Time-steps for Unsupervised Representation Learningcited by 36
  18. 20193D Shape Completion with Multi-view Consistent Inferencecited by 22
  19. 2025CASteer: Cross-Attention Steering for Controllable Concept Erasurecited by 16
  20. 2020Multi-View Consistency Loss for Improved Single-Image 3D Reconstruction of Clothed Peoplecited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Layout-to-Image Generation with Localized Descriptions using ControlNet with Cross-Attention Control
  2. 2026Improving Controllable Generation: Faster Training and Better Performance via x0-Supervision
  3. 2024Layout Control and Semantic Guidance with Attention Loss Backward for T2I Diffusion Model
  4. 2024Prompt-Consistency Image Generation (PCIG): A Unified Framework Integrating LLMs, Knowledge Graphs, and Controllable Diffusion Models
  5. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  6. 2025Enhancing Image Generation Fidelity via Progressive Prompts
  7. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models
  8. 2025ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention