mlsanityData to 14 Oct 2026

In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer

arXiv 2504.20690 ↗First posted 29 Apr 2025cs.CVcited by 204

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Instruction-based image editing enables precise modifications via natural language prompts, but existing methods face a precision-efficiency tradeoff: fine-tuning demands massive datasets (>10M) and computational resources, while training-free approaches suffer from weak instruction comprehension. We address this by proposing ICEdit, which leverages the inherent comprehension and generation abilities of large-scale Diffusion Transformers (DiTs) through three key innovations: (1) An in-context editing paradigm without architectural modifications; (2) Minimal parameter-efficient fine-tuning for quality improvement; (3) Early Filter Inference-Time Scaling, which uses VLMs to select high-quality noise samples for efficiency. Experiments show that ICEdit achieves state-of-the-art editing performance with only 0.1% of the training data and 1% trainable parameters compared to previous methods. Our approach establishes a new paradigm for balancing precision and efficiency in instructional image editing. Codes and demos can be found in https://river-zhang.github.io/ICEdit-gh-pages/.

Key facts. Cited by 204 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 64 of its 75 references that are in the map. First posted on arXiv 29 Apr 2025.

Builds on

64 of 75 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Stable Flow: Vital Layers for Training-Free Image Editing21 Nov
  2. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision11 Nov
  3. 2024SeedEdit: Align Image Re-Generation to Image Editing11 Nov
  4. 2024Taming Rectified Flow for Inversion and Editing7 Nov
  5. 2024In-Context LoRA for Diffusion Transformers31 Oct
  6. 2024Group Diffusion Transformers are Unsupervised Multitask Learners19 Oct
  7. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer30 Sep
  8. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models11 Dec
  9. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code2 Oct
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  11. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov

Other references · 53

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  11. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
38 more
  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  5. 2024Mixtral of Expertscited by 1,823
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  7. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  8. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  9. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  10. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  11. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  12. 2024One Step Diffusion via Shortcut Modelscited by 391
  13. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  14. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  15. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  16. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
  17. 2025Transfer between Modalities with MetaQueriescited by 236
  18. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  19. 2024FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Modelscited by 217
  20. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  21. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185
  22. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  23. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  24. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
  25. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 133
  26. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  27. 2024EraseAnything: Enabling Concept Erasure in Rectified Flow Transformerscited by 95
  28. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  29. 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 92
  30. 2024LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMscited by 84
  31. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  32. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 58
  33. 2025KV-Edit: Training-Free Image Editing for Precise Background Preservationcited by 49
  34. 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
  35. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 37
  36. 2024MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesiscited by 35
  37. 2020Multi-modal Attention for Speech Emotion Recognitioncited by 29
  38. 2024InsightEdit: Towards Better Instruction Following for Image Editingcited by 17

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 87; 2026: 117.

204 papers.

Sort:

By area:

  1. 2025Qwen-Image Technical Reportcited by 1,044
  2. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  3. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  4. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  5. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  6. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 132
  7. 2025LongCat-Image Technical Reportcited by 100
  8. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 79
  9. 2025Ovis-U1 Technical Reportcited by 67
  10. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 66
  11. 2025Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editingcited by 61
  12. 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 54
38 more
  1. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 54
  2. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  3. 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 49
  4. 2025FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editingcited by 46
  5. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 44
  6. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  7. 2026FireRed-Image-Edit-1.0 Technical Reportcited by 36
  8. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 32
  9. 2026HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformercited by 32
  10. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 32
  11. 2025ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulationcited by 29
  12. 2025IMAGHarmony: Controllable Image Editing with Consistent Object Quantity and Layoutcited by 28
  13. 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
  14. 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 25
  15. 2025ReasonEdit: Towards Reasoning-Enhanced Image Editing Modelscited by 25
  16. 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 25
  17. 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 23
  18. 2025Visual Autoregressive Modeling for Instruction-Guided Image Editingcited by 22
  19. 2025ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependenciescited by 21
  20. 2025DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformerscited by 20
  21. 2026Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Toolingcited by 19
  22. 2025Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editingcited by 18
  23. 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generationcited by 18
  24. 2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
  25. 2025Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generationcited by 17
  26. 2025MIRA: Multimodal Iterative Reasoning Agent for Image Editingcited by 17
  27. 2025DreamPoster: A Unified Framework for Image-Conditioned Generative Poster Designcited by 16
  28. 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editingcited by 15
  29. 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
  30. 2025SliderEdit: Continuous Image Editing with Fine-Grained Instruction Controlcited by 14
  31. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
  32. 2026NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generationcited by 14
  33. 2026Diffusion Probe: Generated Image Result Prediction Using CNN Probescited by 14
  34. 2025EasyV2V: A High-quality Instruction-based Video Editing Frameworkcited by 13
  35. 2025ThinkGen: Generalized Thinking for Visual Generationcited by 13
  36. 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 12
  37. 2025HBridge: H-Shape Bridging of Heterogeneous Experts for Unified Multimodal Understanding and Generationcited by 11
  38. 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editingcited by 11

50 most cited of 204 All 204 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing
  2. 2025MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing
  3. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions
  4. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs
  5. 2026EditCrafter: Tuning-free High-Resolution Image Editing via Pretrained Diffusion Model
  6. 2024BrushEdit: All-In-One Image Inpainting and Editing
  7. 2026SpecEdit: Training-Free Acceleration for Diffusion based Image Editing via Semantic Locking
  8. 2026HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing