Data to 7 Oct 2026

OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision

arXiv 2411.07199 ↗First posted 11 Nov 2024cs.CV cs.AIcited by 145

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Instruction-guided image editing methods have demonstrated significant potential by training diffusion models on automatically synthesized or manually annotated image editing pairs. However, these methods remain far from practical, real-life applications. We identify three primary challenges contributing to this gap. Firstly, existing models have limited editing skills due to the biased synthesis process. Secondly, these methods are trained with datasets with a high volume of noise and artifacts. This is due to the application of simple filtering methods like CLIP-score. Thirdly, all these datasets are restricted to a single low resolution and fixed aspect ratio, limiting the versatility to handle real-world use cases. In this paper, we present omniedit, which is an omnipotent editor to handle seven different image editing tasks with any aspect ratio seamlessly. Our contribution is in four folds: (1) omniedit is trained by utilizing the supervision from seven different specialist models to ensure task coverage. (2) we utilize importance sampling based on the scores provided by large multimodal models (like GPT-4o) instead of CLIP-score to improve the data quality. (3) we propose a new editing architecture called EditNet to greatly boost the editing success rate, (4) we provide images with different aspect ratios to ensure that our model can handle any image in the wild. We have curated a test set containing images of different aspect ratios, accompanied by diverse instructions to cover different tasks. Both automatic evaluation and human evaluations demonstrate that omniedit can significantly outperform all the existing models. Our code, dataset and model will be available at https://tiger-ai-lab.github.io/OmniEdit/

Key facts. Cited by 145 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 39 of its 47 references that are in the map. First posted on arXiv 11 Nov 2024.

Builds on

39 of 47 references.

Closest ancestors · 9

Same direction, newest first

Other references · 30

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2020Denoising Diffusion Probabilistic Modelscited by 20,851
2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
2023GPT-4 Technical Reportcited by 18,132
2020Denoising Diffusion Implicit Modelscited by 9,655
2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409

18 more

2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
2023Zero-shot Image-to-Image Translationcited by 503
2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 200
2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 172
2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 81
2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 75
2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
2024Paint by Inpaint: Learning to Add Image Objects by Removing Them Firstcited by 39
2020Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructionscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 84; 2026: 59.

145 papers.

Sort:

By area:

2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 103
2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
2025LongCat-Image Technical Reportcited by 92
2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69

38 more

2025GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Datasetcited by 68
2025Ovis-U1 Technical Reportcited by 65
2025Se~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialistscited by 61
2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 59
2025Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editingcited by 58
2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 53
2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuningcited by 53
2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 53
2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 52
2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 50
2025DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabulariescited by 42
2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generationcited by 39
2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 38
2025Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modelingcited by 35
2025DreamOmni2: Multimodal Instruction-based Editing and Generationcited by 35
2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
2025NoHumansRequired: Autonomous High-Quality Image Editing Triplet Miningcited by 33
2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 31
2026FireRed-Image-Edit-1.0 Technical Reportcited by 31
2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 29
2025textttComplex-Edit: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmarkcited by 25
2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 24
2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 22
2025Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editingcited by 22
2025Visual Autoregressive Modeling for Instruction-Guided Image Editingcited by 21
2025UniVG: A Generalist Diffusion Model for Unified Image Generation and Editingcited by 19
2025SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Modelscited by 19
2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
2025OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditionscited by 17
2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 17
2025Factuality Matters: When Image Generation and Editing Meet Structured Visualscited by 16
2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editingcited by 15
2025EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editingcited by 12
2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 12

50 most cited of 145 All 145 (JSON)

Similar, not cited

Close in wording, no citation either way