# Multi-Reward as Condition for Instruction-based Image Editing

arXiv: https://arxiv.org/abs/2411.04713 · HTML: https://mlsanity.com/paper/2411.04713/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 33 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing](https://mlsanity.com/paper/2404.09990/) (2024)
- [Guiding Instruction-based Image Editing via Multimodal Large Language Models](https://mlsanity.com/paper/2309.17102/) (2023)
- [SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models](https://mlsanity.com/paper/2312.06739/) (2023)
- [HIVE: Harnessing Human Feedback for Instructional Visual Editing](https://mlsanity.com/paper/2303.09618/) (2023)
- [InstructDiffusion: A Generalist Modeling Interface for Vision Tasks](https://mlsanity.com/paper/2309.03895/) (2023)
- [Kosmos-G: Generating Images in Context with Multimodal Large Language Models](https://mlsanity.com/paper/2310.02992/) (2023)
- [SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editing](https://mlsanity.com/paper/2405.04007/) (2024)
- [I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models](https://mlsanity.com/paper/2312.16693/) (2023)
- [Towards Realistic Data Generation for Real-World Super-Resolution](https://mlsanity.com/paper/2406.07255/) (2024)
- [Lightweight Adaptive Feature De-drifting for Compressed Image Classification](https://mlsanity.com/paper/2401.01724/) (2024)
- [Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modeling](https://mlsanity.com/paper/2410.11236/) (2024)
- [Progressive Compositionality in Text-to-Image Generative Models](https://mlsanity.com/paper/2410.16719/) (2024)
- [ByteEdit: Boost, Comply and Accelerate Generative Image Editing](https://mlsanity.com/paper/2404.04860/) (2024)
- [ZeroDiff: Solidified Visual-Semantic Correlation in Zero-Shot Learning](https://mlsanity.com/paper/2406.02929/) (2024)
- [Edit3K: Universal Representation Learning for Video Editing Components](https://mlsanity.com/paper/2403.16048/) (2024)

## Built on by (most cited first, 20 of 20)

- [RewardDance: Reward Scaling in Visual Generation](https://mlsanity.com/paper/2509.08826/) (2025)
- [EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing](https://mlsanity.com/paper/2509.26346/) (2025)
- [Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning](https://mlsanity.com/paper/2505.07538/) (2025)
- [Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2502.11168/) (2025)
- [SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing](https://mlsanity.com/paper/2505.02370/) (2025)
- [WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing](https://mlsanity.com/paper/2512.00387/) (2025)
- [Edicho: Consistent Image Editing in the Wild](https://mlsanity.com/paper/2412.21079/) (2024)
- [Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning](https://mlsanity.com/paper/2506.01480/) (2025)
- [ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning](https://mlsanity.com/paper/2603.08059/) (2026)
- [MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection](https://mlsanity.com/paper/2505.19149/) (2025)
- [ADIEE: Automatic Dataset Creation and Scorer for Instruction-Guided Image Editing Evaluation](https://mlsanity.com/paper/2507.07317/) (2025)
- [CannyEdit: Selective Canny Control and Dual-Prompt Guidance for Training-Free Image Editing](https://mlsanity.com/paper/2508.06937/) (2025)
- [OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment](https://mlsanity.com/paper/2510.10609/) (2025)
- [Toward Trustworthy Portrait Editing: Evaluation of Demographic Misrepresentation in I2I Models](https://mlsanity.com/paper/2602.16149/) (2026)
- [GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing](https://mlsanity.com/paper/2603.21176/) (2026)
- [Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing](https://mlsanity.com/paper/2504.04784/) (2025)
- [InStyle: Instant Appearance Stylization of 3D Shapes](https://mlsanity.com/paper/2512.03683/) (2025)
- [SmartPhotoCrafter: Unified Reasoning, Generation and Optimization for Automatic Photographic Image Editing](https://mlsanity.com/paper/2604.19587/) (2026)
- [CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferences](https://mlsanity.com/paper/2606.00931/) (2026)
- [Semantically Aligned Gradient-Driven Context-Preserving Image Editing](https://mlsanity.com/paper/2609.12691/) (2026)

## Abstract (arXiv)

High-quality training triplets (instruction, original image, edited image) are essential for instruction-based image editing. Predominant training datasets (e.g., InsPix2Pix) are created using text-to-image generative models (e.g., Stable Diffusion, DALL-E) which are not trained for image editing. Accordingly, these datasets suffer from inaccurate instruction following, poor detail preserving, and generation artifacts. In this paper, we propose to address the training data quality issue with multi-perspective reward data instead of refining the ground-truth image quality. 1) we first design a quantitative metric system based on best-in-class LVLM (Large Vision Language Model), i.e., GPT-4o in our case, to evaluate the generation quality from 3 perspectives, namely, instruction following, detail preserving, and generation quality. For each perspective, we collected quantitative score in 0~ 5 and text descriptive feedback on the specific failure points in ground-truth edited images, resulting in a high-quality editing reward dataset, i.e., RewardEdit20K. 2) We further proposed a novel training framework to seamlessly integrate the metric output, regarded as multi-reward, into editing models to learn from the imperfect training triplets. During training, the reward scores and text descriptions are encoded as embeddings and fed into both the latent space and the U-Net of the editing models as auxiliary conditions. 3) We also build a challenging evaluation benchmark with real-world images/photos and diverse editing instructions, named Real-Edit. Experiments indicate that our multi-reward conditioned model outperforms its no-reward counterpart on two popular editing pipelines, i.e., InsPix2Pix and SmartEdit. Code is released at https://github.com/bytedance/Multi-Reward-Editing.