mlsanityData to 7 Oct 2026

Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback

arXiv 2510.16888 ↗First posted 19 Oct 2025cs.CVcited by 78

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Instruction-based image editing has achieved remarkable progress; however, models solely trained via supervised fine-tuning often overfit to annotated patterns, hindering their ability to explore and generalize beyond training distributions. To this end, we introduce Edit-R1, a novel post-training framework for instruction-based image editing based on policy optimization. Specifically, we utilize Diffusion Negative-aware Finetuning (DiffusionNFT), a likelihood-free policy optimization method consistent with the flow matching forward process, thereby enabling the use of higher-order samplers and more efficient training. Another key challenge here is the absence of a universal reward model, resulting from the diverse nature of editing instructions and tasks. To bridge this gap, we employ a Multimodal Large Language Model (MLLM) as a unified, training-free reward model, leveraging its output logits to provide fine-grained feedback. Furthermore, we carefully design a low-variance group filtering mechanism to reduce MLLM scoring noise and stabilize optimization. textttUniWorld-V2, trained with this framework, achieves state-of-the-art results on the ImgEdit and GEdit-Bench benchmarks, scoring 4.49 and 7.83, respectively. Crucially, our framework is model-agnostic, delivering substantial performance gains when applied to diverse base models like Qwen-Image-Edit and FLUX-Kontext, demonstrating its wide applicability. Code and models are publicly available to support further research.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 59 of its 64 references that are in the map. First posted on arXiv 19 Oct 2025.

Builds on

59 of 64 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing30 Sep
  2. 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling28 Sep
  3. 2025RewardDance: Reward Scaling in Visual Generation10 Sep
  4. 2025OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning28 Aug
  5. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning28 Aug
  6. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  7. 2025Unified Reward Model for Multimodal Understanding and Generation7 Mar
  8. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation30 Dec

Other references · 51

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
38 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  2. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  5. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  6. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  7. 2025Qwen-Image Technical Reportcited by 973
  8. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  9. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  10. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  11. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  12. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  13. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  14. 2024OmniGen: Unified Image Generationcited by 388
  15. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  16. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
  17. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  18. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  19. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  20. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  21. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  22. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  23. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  24. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  25. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  26. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  27. 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 83
  28. 2024Large-scale Reinforcement Learning for Diffusion Modelscited by 78
  29. 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Modelscited by 59
  30. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 57
  31. 2025Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Modelcited by 38
  32. 2025Generalist Reward Models: Found Inside Large Language Modelscited by 34
  33. 2025SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Modelscited by 19
  34. 2025AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basincited by 18
  35. 2024HonestLLM: Toward an Honest and Helpful Large Language Modelcited by 17
  36. 2025LeX-Art: Rethinking Text Generation via Scalable High-Quality Data Synthesiscited by 15
  37. 2025Reinforcement Learning with Inverse Rewards for World Model Post-trainingcited by 12
  38. 2025CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-stepcited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 70.

78 papers.

Sort:

By area:

  1. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 247
  2. 2026Helios: Real Real-Time Long Video Generation Modelcited by 56
  3. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  4. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
  5. 2025ReasonEdit: Towards Reasoning-Enhanced Image Editing Modelscited by 24
  6. 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 23
  7. 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 23
  8. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
  9. 2026GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillationcited by 15
  10. 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editingcited by 12
  11. 2026RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Timecited by 11
  12. 2025Scone: Bridging Composition and Distinction in Subject-Driven Image Generation via Unified Understanding-Generation Modelingcited by 10
38 more
  1. 2026ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editingcited by 10
  2. 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 10
  3. 2026GenClaw: Code-Driven Agentic Image Generationcited by 10
  4. 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 9
  5. 2026Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generationcited by 9
  6. 2026TAGRPO: Boosting GRPO on Image-to-Video Generation with Direct Trajectory Alignmentcited by 8
  7. 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedbackcited by 8
  8. 2026Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteriacited by 8
  9. 2026SpatialEdit: Benchmarking Fine-Grained Image Spatial Editingcited by 7
  10. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 6
  11. 2026WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editingcited by 6
  12. 2026Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modelingcited by 6
  13. 2026GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environmentscited by 5
  14. 2026RewardHarness: Self-Evolving Agentic Post-Trainingcited by 5
  15. 2025Are Image-to-Video Models Good Zero-Shot Image Editors?cited by 4
  16. 2026iFSQ: Improving FSQ for Image Generation with 1 Line of Codecited by 4
  17. 2026GenArena: How Can We Achieve Human-Aligned Evaluation for Visual Generation Tasks?cited by 4
  18. 2026CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learningcited by 3
  19. 2026From Statics to Dynamics: Physics-Aware Image Editing with Latent Transition Priorscited by 3
  20. 2026Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Trainingcited by 3
  21. 2026Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shapingcited by 3
  22. 2026AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Rewardcited by 3
  23. 2026Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasonerscited by 3
  24. 2026GMO-E2DIT: Grounded Multi-Operation Editing for E-Commerce Imagescited by 3
  25. 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 3
  26. 2025Vision Bridge Transformer at Scalecited by 2
  27. 2026Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenescited by 2
  28. 2026UniRef-Image-Edit: Towards Scalable and Consistent Multi-Reference Image Editingcited by 2
  29. 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Maskingcited by 2
  30. 2026Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learningcited by 2
  31. 2026When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-Oncited by 1
  32. 2026Advances in GRPO for Generation Models: A Surveycited by 1
  33. 2026Agentic Planning with Reasoning for Image Styling via Offline RLcited by 1
  34. 2026TexEditor: Structure-Preserving Text-Driven Texture Editingcited by 1
  35. 2026GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editingcited by 1
  36. 2026FED-Bench: A Cross-Granular Benchmark for Disentangled Evaluation of Facial Expression Editingcited by 1
  37. 2026AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Controlcited by 1
  38. 2026MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matchingcited by 1

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing
  2. 2025Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing
  3. 2024Specify and Edit: Overcoming Ambiguity in Text-Based Image Editing
  4. 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editing
  5. 2025Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder
  6. 2026Scalable Minimal-Change Learning for Controllable Image Editing
  7. 2025Training-Free Reward-Guided Image Editing via Trajectory Optimal Control
  8. 2025Learning an Image Editing Model without Image Editing Pairs