Refinement via Regeneration: Enlarging Modification Space Boosts Image Refinement in Unified Multimodal Models
Not on the map. Why?
Abstract (arXiv)
Unified multimodal models (UMMs) integrate visual understanding and generation within a single framework. For text-to-image (T2I) tasks, this unified capability allows UMMs to refine outputs after their initial generation, potentially extending the performance upper bound. Current UMM-based refinement methods primarily follow a refinement-via-editing (RvE) paradigm, where UMMs produce editing instructions to modify misaligned regions while preserving aligned content. However, editing instructions often describe prompt-image misalignment only coarsely, leading to incomplete refinement. Moreover, pixel-level preservation, though necessary for editing, unnecessarily restricts the effective modification space for refinement. To address these limitations, we propose Refinement via Regeneration (RvR), a novel framework that reformulates refinement as conditional image regeneration rather than editing. Instead of relying on editing instructions and enforcing strict content preservation, RvR regenerates images conditioned on the target prompt and the semantic tokens of the initial image, enabling more complete semantic alignment with a larger modification space. Extensive experiments demonstrate the effectiveness of RvR, improving Geneval from 0.78 to 0.91, DPGBench from 84.02 to 87.21, and UniGenBench++ from 61.53 to 77.41.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 54 of its 61 references that are in the map. First posted on arXiv 28 Apr 2026.
Builds on
54 of 61 references.
References · 54
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Visual Instruction Tuningcited by 7,792
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2025Qwen2.5-VL Technical Reportcited by 5,418
38 more
- 2024GPT-4o System Cardcited by 4,467
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2015VQA: Visual Question Answeringcited by 2,489
- 2025Qwen-Image Technical Reportcited by 973
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
- 2025Transfer between Modalities with MetaQueriescited by 231
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2024Taming Rectified Flow for Inversion and Editingcited by 200
- 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
- 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 196
- 2025HunyuanImage 3.0 Technical Reportcited by 138
- 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
- 2024ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhancecited by 64
- 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Modelscited by 62
- 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Visioncited by 52
- 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 50
- 2023Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Modelscited by 48
- 2025Interleaving Reasoning for Better Text-to-Image Generationcited by 40
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.