# EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing

arXiv: https://arxiv.org/abs/2509.26346 · HTML: https://mlsanity.com/paper/2509.26346/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 44 of its 55 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 30 of 44)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Step1X-Edit: A Practical Framework for General Image Editing](https://mlsanity.com/paper/2504.17761/) (2025)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Emu Edit: Precise Image Editing via Recognition and Generation Tasks](https://mlsanity.com/paper/2311.10089/) (2023)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [UltraEdit: Instruction-based Fine-Grained Image Editing at Scale](https://mlsanity.com/paper/2407.05282/) (2024)
- [EDICT: Exact Diffusion Inversion via Coupled Transformations](https://mlsanity.com/paper/2211.12446/) (2022)
- [OmniGen2: Towards Instruction-Aligned Multimodal Generation](https://mlsanity.com/paper/2506.18871/) (2025)
- [AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea](https://mlsanity.com/paper/2411.15738/) (2024)
- [HPSv3: Towards Wide-Spectrum Human Preference Score](https://mlsanity.com/paper/2508.03789/) (2025)
- [VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation](https://mlsanity.com/paper/2312.14867/) (2023)
- [HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing](https://mlsanity.com/paper/2404.09990/) (2024)

## Built on by (most cited first, 30 of 62)

- [Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback](https://mlsanity.com/paper/2510.16888/) (2025)
- [Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editing](https://mlsanity.com/paper/2510.19808/) (2025)
- [BLIP3o-NEXT: Next Frontier of Native Image Generation](https://mlsanity.com/paper/2510.15857/) (2025)
- [EditThinker: Unlocking Iterative Reasoning for Any Image Editor](https://mlsanity.com/paper/2512.05965/) (2025)
- [Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing](https://mlsanity.com/paper/2512.17909/) (2025)
- [Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image](https://mlsanity.com/paper/2512.16899/) (2025)
- [GEditBench v2: A Human-Aligned Benchmark for General Image Editing](https://mlsanity.com/paper/2603.28547/) (2026)
- [RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time](https://mlsanity.com/paper/2604.11626/) (2026)
- [PromptRL: Prompt Matters in RL for Flow-Based Image Generation](https://mlsanity.com/paper/2602.01382/) (2026)
- [Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation](https://mlsanity.com/paper/2603.12247/) (2026)
- [Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria](https://mlsanity.com/paper/2605.08354/) (2026)
- [Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey](https://mlsanity.com/paper/2510.01925/) (2025)
- [PaCo-RL: Advancing Reinforcement Learning for Consistent Image Generation with Pairwise Reward Modeling](https://mlsanity.com/paper/2512.04784/) (2025)
- [SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning](https://mlsanity.com/paper/2602.07458/) (2026)
- [WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing](https://mlsanity.com/paper/2603.11593/) (2026)
- [SenseNova-U1.5: Towards Native Unified Visual Intelligence](https://mlsanity.com/paper/2609.11929/) (2026)
- [Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling](https://mlsanity.com/paper/2605.13062/) (2026)
- [Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling](https://mlsanity.com/paper/2604.28185/) (2026)
- [RewardHarness: Self-Evolving Agentic Post-Training](https://mlsanity.com/paper/2605.08703/) (2026)
- [GenArena: How Can We Achieve Human-Aligned Evaluation for Visual Generation Tasks?](https://mlsanity.com/paper/2602.06013/) (2026)
- [EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing](https://mlsanity.com/paper/2603.14916/) (2026)
- [DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editing](https://mlsanity.com/paper/2603.28713/) (2026)
- [VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects](https://mlsanity.com/paper/2604.16272/) (2026)
- [Image Diffusion Preview with Consistency Solver](https://mlsanity.com/paper/2512.13592/) (2025)
- [Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Models](https://mlsanity.com/paper/2602.07533/) (2026)
- [DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editing](https://mlsanity.com/paper/2604.25477/) (2026)
- [AesRM: Improving Video Aesthetics with Expert-Level Feedback](https://mlsanity.com/paper/2604.28078/) (2026)
- [CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2605.25378/) (2026)
- [GMO-E2DIT: Grounded Multi-Operation Editing for E-Commerce Images](https://mlsanity.com/paper/2607.00920/) (2026)
- [DetailAnywhere: Fashion Detail Generation via Cross-Modal Feature Alignment Distillation](https://mlsanity.com/paper/2607.02220/) (2026)

## Abstract (arXiv)

Recently, we have witnessed great progress in image editing with natural language instructions. Several closed-source models like GPT-Image-1, Seedream, and Google-Nano-Banana have shown highly promising progress. However, the open-source models are still lagging. The main bottleneck is the lack of a reliable reward model to scale up high-quality synthetic training data. To address this critical bottleneck, we built EditReward, trained with our new large-scale human preference dataset, meticulously annotated by trained experts following a rigorous protocol containing over 200K preference pairs. EditReward demonstrates superior alignment with human preferences in instruction-guided image editing tasks. Experiments show that EditReward achieves state-of-the-art human correlation on established benchmarks such as GenAI-Bench, AURORA-Bench, ImagenHub, and our new EditReward-Bench, outperforming a wide range of VLM-as-judge models. Furthermore, we use EditReward to select a high-quality subset from the existing noisy ShareGPT-4o-Image dataset. We train Step1X-Edit on the selected subset, which shows significant improvement over training on the full set. This demonstrates EditReward's ability to serve as a reward model to scale up high-quality training data for image editing. Furthermore, its strong alignment suggests potential for advanced applications like reinforcement learning-based post-training and test-time scaling of image editing models. EditReward with its training dataset will be released to help the community build more high-quality image editing training datasets.