# Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback

arXiv: https://arxiv.org/abs/2510.16888 · HTML: https://mlsanity.com/paper/2510.16888/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 59 of its 64 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 30 of 59)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Flow-GRPO: Training Flow Matching Models via Online RL](https://mlsanity.com/paper/2505.05470/) (2025)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [Step1X-Edit: A Practical Framework for General Image Editing](https://mlsanity.com/paper/2504.17761/) (2025)
- [DanceGRPO: Unleashing GRPO on Visual Generation](https://mlsanity.com/paper/2505.07818/) (2025)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding](https://mlsanity.com/paper/2405.08748/) (2024)
- [UltraEdit: Instruction-based Fine-Grained Image Editing at Scale](https://mlsanity.com/paper/2407.05282/) (2024)

## Built on by (most cited first, 30 of 79)

- [Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer](https://mlsanity.com/paper/2511.22699/) (2025)
- [Helios: Real Real-Time Long Video Generation Model](https://mlsanity.com/paper/2603.04379/) (2026)
- [Beyond Language Modeling: An Exploration of Multimodal Pretraining](https://mlsanity.com/paper/2603.03276/) (2026)
- [SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture](https://mlsanity.com/paper/2605.12500/) (2026)
- [InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing](https://mlsanity.com/paper/2603.09877/) (2026)
- [UniREditBench: A Unified Reasoning-based Image Editing Benchmark](https://mlsanity.com/paper/2511.01295/) (2025)
- [ReasonEdit: Towards Reasoning-Enhanced Image Editing Models](https://mlsanity.com/paper/2511.22625/) (2025)
- [EditThinker: Unlocking Iterative Reasoning for Any Image Editor](https://mlsanity.com/paper/2512.05965/) (2025)
- [GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation](https://mlsanity.com/paper/2605.21605/) (2026)
- [RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time](https://mlsanity.com/paper/2604.11626/) (2026)
- [ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing](https://mlsanity.com/paper/2601.03467/) (2026)
- [How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing](https://mlsanity.com/paper/2602.01851/) (2026)
- [JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2605.04128/) (2026)
- [GenClaw: Code-Driven Agentic Image Generation](https://mlsanity.com/paper/2605.30248/) (2026)
- [Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward](https://mlsanity.com/paper/2511.20561/) (2025)
- [Scone: Bridging Composition and Distinction in Subject-Driven Image Generation via Unified Understanding-Generation Modeling](https://mlsanity.com/paper/2512.12675/) (2025)
- [TAGRPO: Boosting GRPO on Image-to-Video Generation with Direct Trajectory Alignment](https://mlsanity.com/paper/2601.05729/) (2026)
- [SpatialEdit: Benchmarking Fine-Grained Image Spatial Editing](https://mlsanity.com/paper/2604.04911/) (2026)
- [PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback](https://mlsanity.com/paper/2602.12127/) (2026)
- [Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation](https://mlsanity.com/paper/2603.12247/) (2026)
- [Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria](https://mlsanity.com/paper/2605.08354/) (2026)
- [LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models](https://mlsanity.com/paper/2602.14147/) (2026)
- [WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing](https://mlsanity.com/paper/2603.11593/) (2026)
- [SenseNova-U1.5: Towards Native Unified Visual Intelligence](https://mlsanity.com/paper/2609.11929/) (2026)
- [Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling](https://mlsanity.com/paper/2605.13062/) (2026)
- [GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments](https://mlsanity.com/paper/2602.09007/) (2026)
- [Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training](https://mlsanity.com/paper/2603.25706/) (2026)
- [RewardHarness: Self-Evolving Agentic Post-Training](https://mlsanity.com/paper/2605.08703/) (2026)
- [Are Image-to-Video Models Good Zero-Shot Image Editors?](https://mlsanity.com/paper/2511.19435/) (2025)
- [iFSQ: Improving FSQ for Image Generation with 1 Line of Code](https://mlsanity.com/paper/2601.17124/) (2026)

## Abstract (arXiv)

Instruction-based image editing has achieved remarkable progress; however, models solely trained via supervised fine-tuning often overfit to annotated patterns, hindering their ability to explore and generalize beyond training distributions. To this end, we introduce Edit-R1, a novel post-training framework for instruction-based image editing based on policy optimization. Specifically, we utilize Diffusion Negative-aware Finetuning (DiffusionNFT), a likelihood-free policy optimization method consistent with the flow matching forward process, thereby enabling the use of higher-order samplers and more efficient training. Another key challenge here is the absence of a universal reward model, resulting from the diverse nature of editing instructions and tasks. To bridge this gap, we employ a Multimodal Large Language Model (MLLM) as a unified, training-free reward model, leveraging its output logits to provide fine-grained feedback. Furthermore, we carefully design a low-variance group filtering mechanism to reduce MLLM scoring noise and stabilize optimization. textttUniWorld-V2, trained with this framework, achieves state-of-the-art results on the ImgEdit and GEdit-Bench benchmarks, scoring 4.49 and 7.83, respectively. Crucially, our framework is model-agnostic, delivering substantial performance gains when applied to diverse base models like Qwen-Image-Edit and FLUX-Kontext, demonstrating its wide applicability. Code and models are publicly available to support further research.