mlsanityData to 14 Oct 2026

EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling

arXiv 2509.23909 ↗First posted 28 Sep 2025cs.CVcited by 64

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multiple samples to produce a desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of a high-fidelity, efficient reward signal. In this work, we present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model. We first introduce EditReward-Bench, a comprehensive benchmark to systematically evaluate reward models on editing quality. Building on this benchmark, we develop EditScore, a series of reward models (7B-72B) for evaluating the quality of instruction-guided image editing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image editing. Our experiments show that, while even the largest open-source VLMs fail to provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, results in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that a high-fidelity, domain-specialized reward model is the key to unlocking the full potential of RL in this domain.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 46 of its 58 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

46 of 58 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025RewardDance: Reward Scaling in Visual Generation10 Sep
  2. 2025OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning28 Aug
  3. 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Models6 Aug
  4. 2025HPSv3: Towards Wide-Spectrum Human Preference Score5 Aug
  5. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  6. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning6 May
  7. 2025Unified Reward Model for Multimodal Understanding and Generation7 Mar
  8. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference25 Mar
  9. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editing16 Mar

Other references · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  12. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
25 more
  1. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  2. 2025Qwen-Image Technical Reportcited by 1,044
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  4. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  5. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 939
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  7. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  8. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  9. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  10. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  11. 2024OmniGen: Unified Image Generationcited by 394
  12. 2017Deep Reinforcement Learning: An Overviewcited by 333
  13. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  14. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
  15. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  16. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  17. 2025Seedream 3.0 Technical Reportcited by 154
  18. 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 106
  19. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 82
  20. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 77
  21. 2025What Makes a Reward Model a Good Teacher? An Optimization Perspectivecited by 73
  22. 2025Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Modelcited by 41
  23. 2024ByteEdit: Boost, Comply and Accelerate Generative Image Editingcited by 10
  24. 2025The Promise of RL for Autoregressive Image Editingcited by 6
  25. 2025ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 51.

64 papers.

Sort:

By area:

  1. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 79
  2. 2026Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidancecited by 36
  3. 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 25
  4. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Imagecited by 20
  5. 2025MIRA: Multimodal Iterative Reasoning Agent for Image Editingcited by 17
  6. 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimizationcited by 16
  7. 2026GEditBench v2: A Human-Aligned Benchmark for General Image Editingcited by 14
  8. 2025ThinkGen: Generalized Thinking for Visual Generationcited by 13
  9. 2026Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generationcited by 9
  10. 2026Leveraging Verifier-Based Reinforcement Learning in Image Editingcited by 9
  11. 2026Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteriacited by 9
  12. 2025Group Relative Attention Guidance for Image Editingcited by 8
38 more
  1. 2025UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learningcited by 8
  2. 2025PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modelingcited by 7
  3. 2025MotionEdit: Benchmarking and Learning Motion-Centric Image Editingcited by 7
  4. 2026SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoningcited by 7
  5. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 7
  6. 2026RPiAE: A Representation-Pivoted Autoencoder Enhancing Both Image Generation and Editingcited by 7
  7. 2026Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modelingcited by 6
  8. 2025ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinementcited by 5
  9. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  10. 2026GenArena: How Can We Achieve Human-Aligned Evaluation for Visual Generation Tasks?cited by 4
  11. 2026EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editingcited by 4
  12. 2025Image Diffusion Preview with Consistency Solvercited by 3
  13. 2026Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Modelscited by 3
  14. 2026CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learningcited by 3
  15. 2026CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructionscited by 3
  16. 2026DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editingcited by 3
  17. 2026DetailAnywhere: Fashion Detail Generation via Cross-Modal Feature Alignment Distillationcited by 3
  18. 2026Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenescited by 2
  19. 2026Advances in GRPO for Generation Models: A Surveycited by 2
  20. 2026Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editingcited by 2
  21. 2026ParetoSlider: Diffusion Models Post-Training for Continuous Reward Controlcited by 2
  22. 2026SpatialFlow-GRPO: Where Spatial Credit Drives Image Editingcited by 2
  23. 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editingcited by 2
  24. 2026RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editingcited by 2
  25. 2025MagicWand: A Universal Agent for Generation and Evaluation Aligned with User Preferencecited by 1
  26. 2026BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modelingcited by 1
  27. 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptationcited by 1
  28. 2026EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinementcited by 1
  29. 2026From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editingcited by 1
  30. 2026MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matchingcited by 1
  31. 2026Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editingcited by 1
  32. 2026Evaluation-Verification Reward for Consistent Multi-Reference Image Editingcited by 1
  33. 2026MIEScore: Human-Aligned Evaluation for Multi-Source Image Editingcited by 1
  34. 2026Can We Perform Online RL for Image Editing without Editing Rewards?cited by 1
  35. 2026Catalyst4D: High-Fidelity 3D-to-4D Scene Editing via Dynamic Propagationcited by 0
  36. 2026MIRAGE: Benchmarking and Aligning Multi-Instance Image Editingcited by 0
  37. 2026EditCaption: Human-Refined SFT and HAE-DPO for Image Editing Instruction Synthesiscited by 0
  38. 2026Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessmentcited by 0

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward
  2. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing
  3. 2024Multi-Reward as Condition for Instruction-based Image Editing
  4. 2026FASTER: Fast Adjoint Stochastic Transport for Endpoint Refinement in Reward-Guided Image Editing
  5. 2025Training-Free Reward-Guided Image Editing via Trajectory Optimal Control
  6. 2025Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder
  7. 2026ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing
  8. 2026HP-Edit: A Human-Preference Post-Training Framework for Image Editing