# Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing arXiv: https://arxiv.org/abs/2409.01322 ยท HTML: https://mlsanity.com/paper/2409.01322/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 24 of its 28 references that are in the map. - First posted on arXiv 2 Sep 2024. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 24 of 24) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022) - [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022) - [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021) - [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022) - [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022) - [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019) - [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022) - [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023) - [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021) - [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023) - [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023) - [EDICT: Exact Diffusion Inversion via Coupled Transformations](https://mlsanity.com/paper/2211.12446/) (2022) - [Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Models](https://mlsanity.com/paper/2305.16807/) (2023) - [SINE: SINgle Image Editing with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.04489/) (2022) - [Effective Real Image Editing with Accelerated Iterative Diffusion Inversion](https://mlsanity.com/paper/2309.04907/) (2023) ## Built on by (most cited first, 17 of 17) - [PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing](https://mlsanity.com/paper/2410.04844/) (2024) - [MaterialFusion: High-Quality, Zero-Shot, and Controllable Material Transfer with Diffusion Models](https://mlsanity.com/paper/2502.06606/) (2025) - [MoEdit: On Learning Quantity Perception for Multi-object Image Editing](https://mlsanity.com/paper/2503.10112/) (2025) - [FADE: Frequency-Aware Diffusion Model Factorization for Video Editing](https://mlsanity.com/paper/2506.05934/) (2025) - [ERDDCI: Exact Reversible Diffusion via Dual-Chain Inversion for High-Quality Image Editing](https://mlsanity.com/paper/2410.14247/) (2024) - [EmoAgent: A Multi-Agent Framework for Diverse Affective Image Manipulation](https://mlsanity.com/paper/2503.11290/) (2025) - [CAMEO: A Conditional and Quality-Aware Multi-Agent Image Editing Orchestrator](https://mlsanity.com/paper/2604.03156/) (2026) - [CPAM: Context-Preserving Adaptive Manipulation for Zero-Shot Real Image Editing](https://mlsanity.com/paper/2506.18438/) (2025) - [Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model](https://mlsanity.com/paper/2504.05594/) (2025) - [Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency Models](https://mlsanity.com/paper/2506.19103/) (2025) - [HeadRouter: A Training-free Image Editing Framework for MM-DiTs by Adaptively Routing Attention Heads](https://mlsanity.com/paper/2411.15034/) (2024) - [Unified Text-Image-to-Video Generation: A Training-Free Approach to Flexible Visual Conditioning](https://mlsanity.com/paper/2505.20629/) (2025) - [FastFace: Tuning Identity Preservation in Distilled Diffusion via Guidance and Attention](https://mlsanity.com/paper/2505.21144/) (2025) - [RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generation](https://mlsanity.com/paper/2507.02792/) (2025) - [FROMAT: Multiview Material Appearance Transfer via Few-Shot Self-Attention Adaptation](https://mlsanity.com/paper/2512.09617/) (2025) - [When One Modality Rules Them All: Backdoor Modality Collapse in Multimodal Diffusion Models](https://mlsanity.com/paper/2603.06508/) (2026) - [Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications](https://mlsanity.com/paper/2607.25642/) (2026) ## Abstract (arXiv) Despite recent advances in large-scale text-to-image generative models, manipulating real images with these models remains a challenging problem. The main limitations of existing editing methods are that they either fail to perform with consistent quality on a wide range of image edits or require time-consuming hyperparameter tuning or fine-tuning of the diffusion model to preserve the image-specific appearance of the input image. We propose a novel approach that is built upon a modified diffusion sampling process via the guidance mechanism. In this work, we explore the self-guidance technique to preserve the overall structure of the input image and its local regions appearance that should not be edited. In particular, we explicitly introduce layout-preserving energy functions that are aimed to save local and global structures of the source image. Additionally, we propose a noise rescaling mechanism that allows to preserve noise distribution by balancing the norms of classifier-free guidance and our proposed guiders during generation. Such a guiding approach does not require fine-tuning the diffusion model and exact inversion process. As a result, the proposed method provides a fast and high-quality editing mechanism. In our experiments, we show through human evaluation and quantitative analysis that the proposed method allows to produce desired editing which is more preferable by humans and also achieves a better trade-off between editing quality and preservation of the original image. Our code is available at https://github.com/MACderRu/Guide-and-Rescale.