mlsanityData to 7 Oct 2026

OmniGen: Unified Image Generation

arXiv 2409.11340 ↗First posted 17 Sep 2024cs.CV cs.AIcited by 388

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

The emergence of Large Language Models (LLMs) has unified language generation tasks and revolutionized human-machine interaction. However, in the realm of image generation, a unified model capable of handling various tasks within a single framework remains largely unexplored. In this work, we introduce OmniGen, a new diffusion model for unified image generation. OmniGen is characterized by the following features: 1) Unification: OmniGen not only demonstrates text-to-image generation capabilities but also inherently supports various downstream tasks, such as image editing, subject-driven generation, and visual-conditional generation. 2) Simplicity: The architecture of OmniGen is highly simplified, eliminating the need for additional plugins. Moreover, compared to existing diffusion models, it is more user-friendly and can complete complex tasks end-to-end through instructions without the need for extra intermediate steps, greatly simplifying the image generation workflow. 3) Knowledge Transfer: Benefit from learning in a unified format, OmniGen effectively transfers knowledge across different tasks, manages unseen tasks and domains, and exhibits novel capabilities. We also explore the model's reasoning capabilities and potential applications of the chain-of-thought mechanism. This work represents the first attempt at a general-purpose image generation model, and we will release our resources at https://github.com/VectorSpaceLab/OmniGen to foster future advancements.

Key facts. Cited by 388 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 71 of its 80 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

71 of 80 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback11 Apr
  2. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  3. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  4. 2023InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists30 Sep
  5. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  6. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  7. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  8. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  9. 2023DataComp: In search of the next generation of multimodal datasets27 Apr
  10. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  12. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
4 more
  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  2. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 55

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
38 more
  1. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  2. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  3. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  4. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  5. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  6. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  7. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  8. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  9. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  10. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  11. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  12. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  13. 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 720
  14. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  15. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  16. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  17. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  18. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  19. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  20. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  21. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  22. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  23. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  24. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  25. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  26. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Modelscited by 204
  27. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  28. 2023InstructDiffusion: A Generalist Modeling Interface for Vision Taskscited by 162
  29. 2019FSS-1000: A 1000-Class Dataset for Few-Shot Segmentationcited by 160
  30. 2022Learning Enriched Features for Fast Image Restoration and Enhancementcited by 150
  31. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 143
  32. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  33. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  34. 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 130
  35. 2022High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditionscited by 129
  36. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  37. 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
  38. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 82

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 217; 2026: 143.

388 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  4. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  5. 2025VACE: All-in-One Video Creation and Editingcited by 467
  6. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  7. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  8. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  9. 2025Transfer between Modalities with MetaQueriescited by 231
  10. 2025Video models are zero-shot learners and reasonerscited by 208
  11. 2024In-Context LoRA for Diffusion Transformerscited by 155
  12. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
38 more
  1. 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
  2. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  3. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  4. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  5. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  6. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  7. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
  8. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  9. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  10. 2025Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generationcited by 95
  11. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  12. 2025LongCat-Image Technical Reportcited by 92
  13. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  14. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  15. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
  16. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  17. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  18. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 78
  19. 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
  20. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  21. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69
  22. 2025Ovis-U1 Technical Reportcited by 65
  23. 2024ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhancecited by 64
  24. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 64
  25. 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modelingcited by 61
  26. 2025Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizablecited by 58
  27. 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 55
  28. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 55
  29. 2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 53
  30. 2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuningcited by 53
  31. 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 53
  32. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 52
  33. 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Visioncited by 52
  34. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  35. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 50
  36. 2024ControlAR: Controllable Image Generation with Autoregressive Modelscited by 48
  37. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 45
  38. 2025UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editingcited by 43

50 most cited of 388 All 388 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025OmniGenBench: A Benchmark for Omnipotent Multimodal Generation across 50+ Tasks
  2. 2025Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generation
  3. 2026Mural: Transferring LLM knowledge to image generation via Mixture-of-Transformers
  4. 2025UniFusion: Vision-Language Model as Unified Encoder in Image Generation
  5. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
  6. 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generation
  7. 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation
  8. 2024UNIMO-G: Unified Image Generation through Multimodal Conditional Diffusion