mlsanityData to 14 Oct 2026

UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation

arXiv 2506.03147 ↗First posted 3 Jun 2025cs.CV cs.AI cs.CLcited by 66

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although existing unified models achieve strong performance in vision-language understanding and text-to-image generation, they remain limited in addressing image perception and manipulation -- capabilities increasingly demanded in practical applications. Recently, OpenAI introduced the powerful GPT-4o-Image model, which showcases advanced capabilities in comprehensive image perception and manipulation, sparking widespread interest. Through carefully designed experiments, we observe that GPT-4o-Image likely relies on semantic encoders rather than VAEs for feature extraction, despite VAEs being commonly regarded as crucial for image manipulation tasks. Inspired by this insight, we propose UniWorld-V1, a unified generative framework built upon semantic features extracted from powerful multimodal large language models and contrastive semantic encoders. Using only 2.7M training data, UniWorld-V1 achieves impressive performance across diverse tasks, including image understanding, generation, manipulation, and perception. We fully open-source the UniWorld-V1 framework, including model weights, training and evaluation scripts, and datasets to promote reproducibility and further research.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 53 of its 57 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

53 of 57 references.

References · 53

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
38 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  7. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  8. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  9. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  10. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  11. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  12. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  13. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  14. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  15. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  16. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  17. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  18. 2024OmniGen: Unified Image Generationcited by 394
  19. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  20. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 370
  21. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  22. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  23. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  24. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  25. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
  26. 2025Transfer between Modalities with MetaQueriescited by 236
  27. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  28. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  29. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  30. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 204
  31. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
  32. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
  33. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 116
  34. 2020Improving Image Captioning with Better Use of Captionscited by 84
  35. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  36. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 58
  37. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  38. 2025OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Datacited by 32

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 50.

66 papers.

Sort:

By area:

  1. 2025Qwen-Image Technical Reportcited by 1,044
  2. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  3. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  4. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36
  5. 2025ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulationcited by 29
  6. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 28
  7. 2025ReasonEdit: Towards Reasoning-Enhanced Image Editing Modelscited by 25
  8. 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editingcited by 15
  9. 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
  10. 2026NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generationcited by 14
  11. 2026DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editingcited by 14
  12. 2025ThinkGen: Generalized Thinking for Visual Generationcited by 13
38 more
  1. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 13
  2. 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editingcited by 12
  3. 2025TBAC-UniImage: Unified Understanding and Generation by Ladder-Side Diffusion Tuningcited by 10
  4. 2026SpatialEdit: Benchmarking Fine-Grained Image Spatial Editingcited by 10
  5. 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 9
  6. 2025UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learningcited by 8
  7. 2026From Scale to Speed: Adaptive Test-Time Scaling for Image Editingcited by 8
  8. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 7
  9. 2026WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editingcited by 7
  10. 2025Few-Shot Synthetic Image Attribution: Identifying Unseen Generators with Limited Samplescited by 6
  11. 2025Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editingcited by 6
  12. 2026Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modelingcited by 6
  13. 2026Dynin-Omni: Omnimodal Unified Large Diffusion Language Modelcited by 5
  14. 2025Are Image-to-Video Models Good Zero-Shot Image Editors?cited by 4
  15. 2026DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editingcited by 4
  16. 2026Representation Forcing for Bottleneck-Free Unified Multimodal Modelscited by 4
  17. 2025PrefGen: Multimodal Preference Learning for Preference-Conditioned Image Generationcited by 3
  18. 2026Generative Visual Chain-of-Thought for Image Editingcited by 3
  19. 2026Meta-CoT: Enhancing Granularity and Generalization in Image Editingcited by 3
  20. 2026DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editingcited by 3
  21. 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoEcited by 3
  22. 2026Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editingcited by 3
  23. 2026Rethinking UMM Visual Generation: Masked Modeling for Efficient Image-Only Pre-trainingcited by 2
  24. 2026UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomycited by 2
  25. 2026ViGoR-Bench: How Far Are Visual Generative Models From Zero-Shot Visual Reasoners?cited by 2
  26. 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Maskingcited by 2
  27. 2026STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generationcited by 2
  28. 2026UniCustom: Unified Visual Conditioning for Multi-Reference Image Generationcited by 2
  29. 2026TextWand: A Unified Framework for Scene Text Editingcited by 2
  30. 2025AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Modelcited by 1
  31. 2026Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matchingcited by 1
  32. 2026VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-Oncited by 1
  33. 2026LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editingcited by 1
  34. 2026MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddingscited by 1
  35. 2026Large Language Models are Universal Reasoners for Visual Generationcited by 1
  36. 2026From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editingcited by 1
  37. 2026SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Modelscited by 1
  38. 2026DynEval: Holistic Evaluations of T2I Generative Models in the Wildcited by 1

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifying
  2. 2025UniFusion: Vision-Language Model as Unified Encoder in Image Generation
  3. 2026UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
  4. 2025UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
  5. 2025UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception
  6. 2025DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies
  7. 2026UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer
  8. 2025On the Limitations of Vision-Language Models in Understanding Image Transforms