mlsanityData to 14 Oct 2026

UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics

arXiv 2412.07774 ↗First posted 10 Dec 2024cs.CVcited by 102

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce UniReal, a unified framework designed to address various image generation and editing tasks. Existing solutions often vary by tasks, yet share fundamental principles: preserving consistency between inputs and outputs while capturing visual variations. Inspired by recent video generation models that effectively balance consistency and variation across frames, we propose a unifying approach that treats image-level tasks as discontinuous video generation. Specifically, we treat varying numbers of input and output images as frames, enabling seamless support for tasks such as image generation, editing, customization, composition, etc. Although designed for image-level tasks, we leverage videos as a scalable source for universal supervision. UniReal learns world dynamics from large-scale videos, demonstrating advanced capability in handling shadows, reflections, pose variation, and object interaction, while also exhibiting emergent capability for novel applications.

Key facts. Cited by 102 papers in the map (data through 14 Oct 2026). Builds on 68 of its 82 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

68 of 82 references.

References · 68

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023Segment Anythingcited by 8,234
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
38 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  4. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  5. 2024Depth Anything V2cited by 1,422
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  8. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  9. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  10. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  11. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  12. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  13. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  14. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  15. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  16. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  17. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  18. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  19. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  20. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  21. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  22. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  23. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  24. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  25. 2024OmniGen: Unified Image Generationcited by 394
  26. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  27. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  28. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  29. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 304
  30. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  31. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  32. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  33. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 212
  34. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
  35. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 202
  36. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 199
  37. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 193
  38. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 64; 2026: 38.

102 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  2. 2025VACE: All-in-One Video Creation and Editingcited by 479
  3. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  4. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 129
  5. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  6. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  7. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 87
  8. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  9. 2025DreamO: A Unified Framework for Image Customizationcited by 83
  10. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  11. 2025Insert Anything: Image Insertion via In-Context Editing in DiTcited by 61
  12. 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 56
38 more
  1. 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 54
  2. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 47
  3. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  4. 2025ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulationcited by 29
  5. 2025RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformerscited by 26
  6. 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixelscited by 24
  7. 2025Personalize Anything for Free with Diffusion Transformercited by 23
  8. 2025OmniVDiff: Omni Controllable Video Diffusion for Generation and Understandingcited by 22
  9. 2025Factuality Matters: When Image Generation and Editing Meet Structured Visualscited by 20
  10. 2025ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motionscited by 19
  11. 2025UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Rewardcited by 19
  12. 2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
  13. 2025RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Modelscited by 16
  14. 2025PICABench: How Far Are We from Physically Realistic Image Editing?cited by 16
  15. 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editingcited by 15
  16. 2025Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Visioncited by 14
  17. 2025Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adaptercited by 13
  18. 2025Precise Object and Effect Removal with Adaptive Target-Aware Attentioncited by 13
  19. 2025OmniTry: Virtual Try-On Anything without Maskscited by 13
  20. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  21. 2025EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editingcited by 12
  22. 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 12
  23. 2025MMGen: Unified Multi-modal Image Generation and Understanding in One Gocited by 11
  24. 2025Flux Already Knows -- Activating Subject-Driven Image Generation without Trainingcited by 11
  25. 2025MUSAR: Exploring Multi-Subject Customization from Single-Subject Dataset via Attention Routingcited by 11
  26. 2025Hallucination at a Glance: Controlled Visual Edits and Fine-Grained Multimodal Learningcited by 11
  27. 2025HBridge: H-Shape Bridging of Heterogeneous Experts for Unified Multimodal Understanding and Generationcited by 11
  28. 2026Group Editing: Edit Multiple Images in One Gocited by 11
  29. 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editingcited by 10
  30. 2025Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editingcited by 10
  31. 2025Image Editing As Programs with Diffusion Modelscited by 10
  32. 2025IC-Custom: Diverse Image Customization via In-Context Learningcited by 10
  33. 2025MultiRef: Controllable Image Generation with Multiple Visual Referencescited by 9
  34. 2025LightFusion: A Light-weighted, Double Fusion Framework for Unified Multimodal Understanding and Generationcited by 9
  35. 2025EditIDv2: Editable ID Customization with Data-Lubricated ID Feature Integration for Text-to-Image Generationcited by 8
  36. 2025SeqTex: Generate Mesh Textures in Video Sequencecited by 7
  37. 2025RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformercited by 6
  38. 2025ID-Crafter: VLM-Grounded Online RL for Compositional Multi-Subject Video Generationcited by 6

50 most cited of 102 All 102 (JSON)