# ReGeneration Learning of Diffusion Models with Rich Prompts for Zero-Shot Image Translation arXiv: https://arxiv.org/abs/2305.04651 ยท HTML: https://mlsanity.com/paper/2305.04651/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 30 of its 34 references that are in the map. - First posted on arXiv 8 May 2023. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022) - [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013) - [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021) - [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022) - [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022) - [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021) - [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022) - [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022) - [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022) - [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021) - [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021) - [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022) - [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020) - [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023) - [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022) - [Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance](https://mlsanity.com/paper/2210.05559/) (2022) - [The Spatially-Correlative Loss for Various Image Translation Tasks](https://mlsanity.com/paper/2104.00854/) (2021) - [Regeneration Learning: A Learning Paradigm for Data Generation](https://mlsanity.com/paper/2301.08846/) (2023) - [Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2204.11018/) (2022) ## Built on by (most cited first, 6 of 6) - [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024) - [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024) - [VINCIE: Unlocking In-context Image Editing from Video](https://mlsanity.com/paper/2506.10941/) (2025) - [Disrupting Diffusion-based Inpainters with Semantic Digression](https://mlsanity.com/paper/2407.10277/) (2024) - [Prompt Augmentation for Self-supervised Text-guided Image Manipulation](https://mlsanity.com/paper/2412.13081/) (2024) - [Multiple Noises in Diffusion Model for Semi-Supervised Multi-Domain Translation](https://mlsanity.com/paper/2309.14394/) (2023) ## Abstract (arXiv) Large-scale text-to-image models have demonstrated amazing ability to synthesize diverse and high-fidelity images. However, these models are often violated by several limitations. Firstly, they require the user to provide precise and contextually relevant descriptions for the desired image modifications. Secondly, current models can impose significant changes to the original image content during the editing process. In this paper, we explore ReGeneration learning in an image-to-image Diffusion model (ReDiffuser), that preserves the content of the original image without human prompting and the requisite editing direction is automatically discovered within the text embedding space. To ensure consistent preservation of the shape during image editing, we propose cross-attention guidance based on regeneration learning. This novel approach allows for enhanced expression of the target domain features while preserving the original shape of the image. In addition, we introduce a cooperative update strategy, which allows for efficient preservation of the original shape of an image, thereby improving the quality and consistency of shape preservation throughout the editing process. Our proposed method leverages an existing pre-trained text-image diffusion model without any additional training. Extensive experiments show that the proposed method outperforms existing work in both real and synthetic image editing.