LooseRoPE: Content-aware Attention Manipulation for Semantic Harmonization
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Recent diffusion-based image editing methods commonly rely on text or high-level instructions to guide the generation process, offering intuitive but coarse control. In contrast, we focus on explicit, prompt-free editing, where the user directly specifies the modification by cropping and pasting an object or sub-object into a chosen location within an image. This operation affords precise spatial and visual control, yet it introduces a fundamental challenge: preserving the identity of the pasted object while harmonizing it with its new context. We observe that attention maps in diffusion-based editing models inherently govern whether image regions are preserved or adapted for coherence. Building on this insight, we introduce LooseRoPE, a saliency-guided modulation of rotational positional encoding (RoPE) that loosens the positional constraints to continuously control the attention field of view. By relaxing RoPE in this manner, our method smoothly steers the model's focus between faithful preservation of the input image and coherent harmonization of the inserted object, enabling a balanced trade-off between identity retention and contextual blending. Our approach provides a flexible and intuitive framework for image editing, achieving seamless compositional results without textual descriptions or complex user input.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 43 of its 49 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
43 of 49 references.
Closest ancestors · 14
Same direction, newest first
- 2025Image Generation from Contextually-Contradictory Prompts2 Jun
- 2025Be Decisive: Noise-Induced Layouts for Multi-Subject Generation27 May
- 2024Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generation25 Mar
- 2024One-Step Image Translation with Text-to-Image Models18 Mar
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
- 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition24 Jul
- 2023Cross-domain Compositing with Pretrained Diffusion Models20 Feb
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
- 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation16 Feb
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
- 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
2 more
Other references · 29
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2025Qwen-Image Technical Reportcited by 1,044
- 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
17 more
- 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
- 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
- 2024Rotary Position Embedding for Vision Transformercited by 236
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
- 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 135
- 2017Deep Image Harmonizationcited by 120
- 2019Improving the Harmony of the Composite Image by Spatial-Separated Attention Modulecited by 73
- 2024IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representationcited by 64
- 2021SSH: A Self-Supervised Framework for Image Harmonizationcited by 60
- 2023Relightful Harmonization: Lighting-aware Portrait Background Replacementcited by 57
- 2024Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videoscited by 38
- 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
- 2024SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual Editingcited by 13
- 2025InstanceGen: Image Generation with Instance-level Instructionscited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025The Devil is in Attention Sharing: Improving Complex Non-rigid Image Editing Faithfulness via Attention Synergy
- 2024Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing
- 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs
- 2025Self-Attention Decomposition For Training Free Diffusion Editing
- 2024Learn Once, Edit Anywhere: Visual Direction Transfer for Diffusion Models
- 2026LAMS-Edit: Latent and Attention Mixing with Schedulers for Improved Content Preservation in Diffusion-Based Image and Style Editing
- 2026Dual-Channel Attention Guidance for Training-Free Image Editing Control in Diffusion Transformers
- 2022The Stable Artist: Steering Semantics in Diffusion Latent Space