mlsanityData to 7 Oct 2026

Refinement via Regeneration: Enlarging Modification Space Boosts Image Refinement in Unified Multimodal Models

arXiv 2604.25636 ↗First posted 28 Apr 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Unified multimodal models (UMMs) integrate visual understanding and generation within a single framework. For text-to-image (T2I) tasks, this unified capability allows UMMs to refine outputs after their initial generation, potentially extending the performance upper bound. Current UMM-based refinement methods primarily follow a refinement-via-editing (RvE) paradigm, where UMMs produce editing instructions to modify misaligned regions while preserving aligned content. However, editing instructions often describe prompt-image misalignment only coarsely, leading to incomplete refinement. Moreover, pixel-level preservation, though necessary for editing, unnecessarily restricts the effective modification space for refinement. To address these limitations, we propose Refinement via Regeneration (RvR), a novel framework that reformulates refinement as conditional image regeneration rather than editing. Instead of relying on editing instructions and enforcing strict content preservation, RvR regenerates images conditioned on the target prompt and the semantic tokens of the initial image, enabling more complete semantic alignment with a larger modification space. Extensive experiments demonstrate the effectiveness of RvR, improving Geneval from 0.78 to 0.91, DPGBench from 84.02 to 87.21, and UniGenBench++ from 61.53 to 77.41.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 54 of its 61 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

54 of 61 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2024The Llama 3 Herd of Modelscited by 14,192
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2025Qwen2.5-VL Technical Reportcited by 5,418
38 more
  1. 2024GPT-4o System Cardcited by 4,467
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  7. 2015VQA: Visual Question Answeringcited by 2,489
  8. 2025Qwen-Image Technical Reportcited by 973
  9. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  11. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  12. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  13. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  14. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  15. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  16. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  17. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  18. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  19. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  20. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  21. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  22. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  23. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  24. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  25. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  26. 2025Transfer between Modalities with MetaQueriescited by 231
  27. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  28. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  29. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  30. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 196
  31. 2025HunyuanImage 3.0 Technical Reportcited by 138
  32. 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
  33. 2024ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhancecited by 64
  34. 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Modelscited by 62
  35. 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Visioncited by 52
  36. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 50
  37. 2023Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Modelscited by 48
  38. 2025Interleaving Reasoning for Better Text-to-Image Generationcited by 40

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.