mlsanityData to 7 Oct 2026

DiT4Edit: Diffusion Transformer for Image Editing

arXiv 2411.03286 ↗First posted 5 Nov 2024cs.CVcited by 100

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Despite recent advances in UNet-based image editing, methods for shape-aware object editing in high-resolution images are still lacking. Compared to UNet, Diffusion Transformers (DiT) demonstrate superior capabilities to effectively capture the long-range dependencies among patches, leading to higher-quality image generation. In this paper, we propose DiT4Edit, the first Diffusion Transformer-based image editing framework. Specifically, DiT4Edit uses the DPM-Solver inversion algorithm to obtain the inverted latents, reducing the number of steps compared to the DDIM inversion algorithm commonly used in UNet-based frameworks. Additionally, we design unified attention control and patches merging, tailored for transformer computation streams. This integration allows our framework to generate higher-quality edited images faster. Our design leverages the advantages of DiT, enabling it to surpass UNet structures in image editing, especially in high-resolution and arbitrary-size images. Extensive experiments demonstrate the strong performance of DiT4Edit across various editing scenarios, highlighting the potential of Diffusion Transformers in supporting image editing.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 36 of its 46 references that are in the map. First posted on arXiv 5 Nov 2024.

Builds on

36 of 46 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Inversion-Free Image Editing with Natural Language7 Dec
  2. 2022The Stable Artist: Steering Semantics in Diffusion Latent Space12 Dec
  3. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  4. 2022Inversion-Based Style Transfer with Diffusion Models23 Nov
  5. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  6. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
17 more
  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  3. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  4. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  5. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  6. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  7. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Modelscited by 127
  8. 2018Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Networkcited by 64
  9. 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
  10. 2024COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editingcited by 59
  11. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  12. 2023On Exact Inversion of DPM-Solverscited by 47
  13. 2022DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylizationcited by 45
  14. 2024Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generationcited by 42
  15. 2024Improving Diffusion-Based Image Synthesis with Context Predictioncited by 32
  16. 2022SimVTP: Simple Video Text Pre-training with Masked Autoencoderscited by 21
  17. 2019End-to-End Time-Lapse Video Synthesis from a Single Outdoor Imagecited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 51; 2026: 44.

100 papers.

Sort:

By area:

  1. 2025Controllable Video Generation: A Surveycited by 93
  2. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
  3. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 61
  4. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 52
  5. 2025KV-Edit: Training-Free Image Editing for Precise Background Preservationcited by 47
  6. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  7. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 41
  8. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  9. 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Trainingcited by 33
  10. 2024MagicQuill: An Intelligent Interactive Image Editing Systemcited by 30
  11. 2025IMAGHarmony: Controllable Image Editing with Consistent Object Quantity and Layoutcited by 28
  12. 2025MaskGWM: A Generalizable Driving World Model with Video Mask Reconstructioncited by 27
38 more
  1. 2025Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region Controlcited by 27
  2. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  3. 2024Rethinking Token-wise Feature Caching: Accelerating Diffusion Transformers with Dual Feature Cachingcited by 26
  4. 2025Edit Transfer: Learning Image Editing via Vision In-Context Relationscited by 23
  5. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  6. 2025Training-Free Text-Guided Image Editing with Visual Autoregressive Modelcited by 21
  7. 2025Visual Autoregressive Modeling for Instruction-Guided Image Editingcited by 21
  8. 2025AE-NeRF: Augmenting Event-Based Neural Radiance Fields for Non-ideal Conditions and Larger Scenecited by 19
  9. 2025DreamPoster: A Unified Framework for Image-Conditioned Generative Poster Designcited by 15
  10. 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editingcited by 15
  11. 2024UniPaint: Unified Space-time Video Inpainting via Mixture-of-Expertscited by 13
  12. 2025TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesiscited by 12
  13. 2025EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editingcited by 12
  14. 2025LazyMAR: Accelerating Masked Autoregressive Models via Feature Cachingcited by 11
  15. 2025SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skippingcited by 11
  16. 2026Group Editing: Edit Multiple Images in One Gocited by 11
  17. 2025Follow-Your-Color: Multi-Instance Sketch Colorizationcited by 10
  18. 2025Image Editing As Programs with Diffusion Modelscited by 10
  19. 2025Follow-Your-Instruction: A Comprehensive MLLM Agent for World Data Synthesiscited by 10
  20. 2025UnicEdit-10M: A Dataset and Benchmark Breaking the Scale-Quality Barrier via Unified Verification for Reasoning-Enriched Editscited by 9
  21. 2026SRA 2: Variational Autoencoder Self-Representation Alignment for Efficient Diffusion Trainingcited by 8
  22. 2024AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Modelscited by 7
  23. 2024Follow-Your-MultiPose: Tuning-Free Multi-Character Text-to-Video Generation via Pose Guidancecited by 7
  24. 2025Enhancing Image Generation Fidelity via Progressive Promptscited by 7
  25. 2025EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editingcited by 7
  26. 2025Describe, Don't Dictate: Semantic Image Editing with Natural Language Intentcited by 6
  27. 2025RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformercited by 6
  28. 2026LiveLight: Real-time Streaming Video Relighting with Interactive Controlcited by 6
  29. 2025The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignmentcited by 5
  30. 2026Embedding-perturbed Exploration Preference Optimization for Flow Modelscited by 5
  31. 2026Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Frameworkcited by 5
  32. 2026I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editingcited by 4
  33. 2026GlyphBanana: Advancing Precise Text Rendering Through Agentic Workflowscited by 4
  34. 2025LDGen: Enhancing Text-to-Image Synthesis via Large Language Model-Driven Language Representationcited by 3
  35. 2025DFVEdit: Conditional Delta Flow Vector for Zero-shot Video Editingcited by 3
  36. 2025LORE: Latent Optimization for Precise Semantic Control in Rectified Flow-based Image Editingcited by 3
  37. 2025MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformercited by 3
  38. 2025Breaking the Vicious Cycle: Coherent 3D Gaussian Splatting from Sparse and Motion-Blurred Viewscited by 3

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression
  2. 2023DiffiT: Diffusion Vision Transformers for Image Generation
  3. 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution
  4. 2025SpotEdit: Selective Region Editing in Diffusion Transformers
  5. 2024TurboEdit: Instant text-based image editing
  6. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code
  7. 2024Stable Flow: Vital Layers for Training-Free Image Editing
  8. 2025Exploring Multimodal Diffusion Transformers for Enhanced Prompt-based Image Editing