mlsanityData to 14 Oct 2026

EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing

arXiv 2509.26346 ↗First posted 30 Sep 2025cs.CV cs.AI cs.CLcited by 62

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Recently, we have witnessed great progress in image editing with natural language instructions. Several closed-source models like GPT-Image-1, Seedream, and Google-Nano-Banana have shown highly promising progress. However, the open-source models are still lagging. The main bottleneck is the lack of a reliable reward model to scale up high-quality synthetic training data. To address this critical bottleneck, we built EditReward, trained with our new large-scale human preference dataset, meticulously annotated by trained experts following a rigorous protocol containing over 200K preference pairs. EditReward demonstrates superior alignment with human preferences in instruction-guided image editing tasks. Experiments show that EditReward achieves state-of-the-art human correlation on established benchmarks such as GenAI-Bench, AURORA-Bench, ImagenHub, and our new EditReward-Bench, outperforming a wide range of VLM-as-judge models. Furthermore, we use EditReward to select a high-quality subset from the existing noisy ShareGPT-4o-Image dataset. We train Step1X-Edit on the selected subset, which shows significant improvement over training on the full set. This demonstrates EditReward's ability to serve as a reward model to scale up high-quality training data for image editing. Furthermore, its strong alignment suggests potential for advanced applications like reinforcement learning-based post-training and test-time scaling of image editing models. EditReward with its training dataset will be released to help the community build more high-quality image editing training datasets.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 55 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

44 of 55 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025HPSv3: Towards Wide-Spectrum Human Preference Score5 Aug
  2. 2025Improving Video Generation with Human Feedback23 Jan
  3. 2023Rich Human Feedback for Text-to-Image Generation15 Dec
  4. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference25 Mar
  5. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editing16 Mar

Other references · 39

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  9. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  11. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
27 more
  1. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  2. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  3. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  4. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  5. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  6. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  7. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  8. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  9. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  10. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  11. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
  12. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  13. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
  14. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  15. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 193
  16. 2025Seedream 3.0 Technical Reportcited by 154
  17. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
  18. 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generationcited by 97
  19. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 82
  20. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 77
  21. 2025GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Datasetcited by 70
  22. 2025Ovis-U1 Technical Reportcited by 67
  23. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 53
  24. 2024Learning Action and Reasoning-Centric Image Editing from Videos and Simulationscited by 32
  25. 2024Multi-Reward as Condition for Instruction-based Image Editingcited by 20
  26. 2025A Unified Agentic Framework for Evaluating Conditional Image Generationcited by 15
  27. 2025ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 52.

62 papers.

Sort:

By area:

  1. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 79
  2. 2025Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editingcited by 61
  3. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 54
  4. 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 25
  5. 2025Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editingcited by 22
  6. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Imagecited by 20
  7. 2026GEditBench v2: A Human-Aligned Benchmark for General Image Editingcited by 14
  8. 2026RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Timecited by 14
  9. 2026PromptRL: Prompt Matters in RL for Flow-Based Image Generationcited by 12
  10. 2026Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generationcited by 9
  11. 2026Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteriacited by 9
  12. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 8
38 more
  1. 2025PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modelingcited by 7
  2. 2026SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoningcited by 7
  3. 2026WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editingcited by 7
  4. 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 7
  5. 2026Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modelingcited by 6
  6. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  7. 2026RewardHarness: Self-Evolving Agentic Post-Trainingcited by 5
  8. 2026GenArena: How Can We Achieve Human-Aligned Evaluation for Visual Generation Tasks?cited by 4
  9. 2026EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editingcited by 4
  10. 2026DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editingcited by 4
  11. 2026VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effectscited by 4
  12. 2025Image Diffusion Preview with Consistency Solvercited by 3
  13. 2026Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Modelscited by 3
  14. 2026DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editingcited by 3
  15. 2026AesRM: Improving Video Aesthetics with Expert-Level Feedbackcited by 3
  16. 2026CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillationcited by 3
  17. 2026GMO-E2DIT: Grounded Multi-Operation Editing for E-Commerce Imagescited by 3
  18. 2026DetailAnywhere: Fashion Detail Generation via Cross-Modal Feature Alignment Distillationcited by 3
  19. 2026The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metriccited by 3
  20. 2025ViDiC: Video Difference Captioningcited by 2
  21. 2026SpatialFlow-GRPO: Where Spatial Credit Drives Image Editingcited by 2
  22. 2026RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editingcited by 2
  23. 2026BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modelingcited by 1
  24. 2026OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolutioncited by 1
  25. 2026Sparkle: Realizing Lively Instruction-Guided Video Background Replacement via Decoupled Guidancecited by 1
  26. 2026EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinementcited by 1
  27. 2026ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learningcited by 1
  28. 2026From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editingcited by 1
  29. 2026MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matchingcited by 1
  30. 2026LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Modelscited by 1
  31. 2026Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-Oncited by 1
  32. 2026MIEScore: Human-Aligned Evaluation for Multi-Source Image Editingcited by 1
  33. 2026Can We Perform Online RL for Image Editing without Editing Rewards?cited by 1
  34. 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 1
  35. 2026Pinterest Canvas: Large-Scale Image Generation at Pinterestcited by 0
  36. 2026Edit-GRPO: A Locality-Preserving Policy Optimization Framework for Image Editingcited by 0
  37. 2026CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferencescited by 0
  38. 2026Stateful Visual Encoders for Vision-Language Modelscited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026HP-Edit: A Human-Preference Post-Training Framework for Image Editing
  2. 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling
  3. 2026EditCaption: Human-Refined SFT and HAE-DPO for Image Editing Instruction Synthesis
  4. 2026Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis
  5. 2026Leveraging Verifier-Based Reinforcement Learning in Image Editing
  6. 2025Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder
  7. 2026Scalable Minimal-Change Learning for Controllable Image Editing
  8. 2026CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning