# A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models arXiv: https://arxiv.org/abs/2406.14555 ยท HTML: https://mlsanity.com/paper/2406.14555/ ## Key facts - Cited by 52 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 1 of its 312 references that are in the map. - First posted on arXiv 20 Jun 2024. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 1 of 1) - [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023) ## Built on by (most cited first, 30 of 52) - [SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model](https://mlsanity.com/paper/2412.04292/) (2024) - [KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models](https://mlsanity.com/paper/2505.16707/) (2025) - [SegPoint: Segment Any Point Cloud via Large Language Model](https://mlsanity.com/paper/2407.13761/) (2024) - [How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?](https://mlsanity.com/paper/2410.17594/) (2024) - [BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanation](https://mlsanity.com/paper/2505.12620/) (2025) - [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024) - [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025) - [RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers](https://mlsanity.com/paper/2502.14377/) (2025) - [CharaConsist: Fine-Grained Consistent Character Generation](https://mlsanity.com/paper/2507.11533/) (2025) - [AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration](https://mlsanity.com/paper/2412.11706/) (2024) - [Free-Form Motion Control: Controlling the 6D Poses of Camera and Objects in Video Generation](https://mlsanity.com/paper/2501.01425/) (2025) - [Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark](https://mlsanity.com/paper/2411.15488/) (2024) - [How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing](https://mlsanity.com/paper/2602.01851/) (2026) - [SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing](https://mlsanity.com/paper/2411.18983/) (2024) - [AnyI2V: Animating Any Conditional Image with Motion Control](https://mlsanity.com/paper/2507.02857/) (2025) - [UnicEdit-10M: A Dataset and Benchmark Breaking the Scale-Quality Barrier via Unified Verification for Reasoning-Enriched Edits](https://mlsanity.com/paper/2512.02790/) (2025) - [SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulation](https://mlsanity.com/paper/2511.16666/) (2025) - [GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation](https://mlsanity.com/paper/2512.17495/) (2025) - [Rethinking Cross-Generator Image Forgery Detection through DINOv3](https://mlsanity.com/paper/2511.22471/) (2025) - [An LLM-LVLM Driven Agent for Iterative and Fine-Grained Image Editing](https://mlsanity.com/paper/2508.17435/) (2025) - [EditInfinity: Image Editing with Binary-Quantized Generative Models](https://mlsanity.com/paper/2510.20217/) (2025) - [GlyphPrinter: Region-Grouped Direct Preference Optimization for Glyph-Accurate Visual Text Rendering](https://mlsanity.com/paper/2603.15616/) (2026) - [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024) - [PVUW 2025 Challenge Report: Advances in Pixel-level Understanding of Complex Videos in the Wild](https://mlsanity.com/paper/2504.11326/) (2025) - [So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection](https://mlsanity.com/paper/2505.18660/) (2025) - [BadPatch: Diffusion-Based Generation of Physical Adversarial Patches](https://mlsanity.com/paper/2412.01440/) (2024) - [Neural-Driven Image Editing](https://mlsanity.com/paper/2507.05397/) (2025) - [LSVOS 2025 Challenge Report: Recent Advances in Complex Video Object Segmentation](https://mlsanity.com/paper/2510.11063/) (2025) - [Mapping the Mind of an Instruction-based Image Editing using SMILE](https://mlsanity.com/paper/2412.16277/) (2024) - [Efficient Task-specific Conditional Diffusion Policies: Shortcut Model Acceleration and SO(3) Optimization](https://mlsanity.com/paper/2504.09927/) (2025) ## Abstract (arXiv) Image editing aims to edit the given synthetic or real image to meet the specific requirements from users. It is widely studied in recent years as a promising and challenging field of Artificial Intelligence Generative Content (AIGC). Recent significant advancement in this field is based on the development of text-to-image (T2I) diffusion models, which generate images according to text prompts. These models demonstrate remarkable generative capabilities and have become widely used tools for image editing. T2I-based image editing methods significantly enhance editing performance and offer a user-friendly interface for modifying content guided by multimodal inputs. In this survey, we provide a comprehensive review of multimodal-guided image editing techniques that leverage T2I diffusion models. First, we define the scope of image editing from a holistic perspective and detail various control signals and editing scenarios. We then propose a unified framework to formalize the editing process, categorizing it into two primary algorithm families. This framework offers a design space for users to achieve specific goals. Subsequently, we present an in-depth analysis of each component within this framework, examining the characteristics and applicable scenarios of different combinations. Given that training-based methods learn to directly map the source image to target one under user guidance, we discuss them separately, and introduce injection schemes of source image in different scenarios. Additionally, we review the application of 2D techniques to video editing, highlighting solutions for inter-frame inconsistency. Finally, we discuss open challenges in the field and suggest potential future research directions. We keep tracing related works at https://github.com/xinchengshuai/Awesome-Image-Editing.