mlsanityData to 7 Oct 2026

HIVE: Harnessing Human Feedback for Instructional Visual Editing

arXiv 2303.09618 ↗First posted 16 Mar 2023cs.CV cs.AI cs.CLcited by 172

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences. We hypothesize that state-of-the-art instructional image editing models, where outputs are generated based on an input image and an editing instruction, could similarly benefit from human feedback, as their outputs may not adhere to the correct instructions and preferences of users. In this paper, we present a novel framework to harness human feedback for instructional visual editing (HIVE). Specifically, we collect human feedback on the edited images and learn a reward function to capture the underlying user preferences. We then introduce scalable diffusion model fine-tuning methods that can incorporate human preferences based on the estimated reward. Besides, to mitigate the bias brought by the limitation of data, we contribute a new 1M training dataset, a 3.6K reward dataset for rewards learning, and a 1K evaluation dataset to boost the performance of instructional image editing. We conduct extensive empirical experiments quantitatively and qualitatively, showing that HIVE is favored over previous state-of-the-art instructional image editing approaches by a large margin.

Key facts. Cited by 172 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 51 of its 64 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

51 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Aligning Text-to-Image Models using Human Feedback23 Feb

Other references · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2017Proximal Policy Optimization Algorithmscited by 14,441
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
38 more
  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  13. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  14. 2020Learning to summarize from human feedbackcited by 2,562
  15. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  16. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  17. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  18. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  19. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  20. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  21. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  22. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  23. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  24. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  25. 2016Generative Adversarial Text to Image Synthesiscited by 735
  26. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  27. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  28. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  29. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  30. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  31. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  32. 2022Blended Latent Diffusioncited by 430
  33. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  34. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  35. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  36. 2022MagicMix: Semantic Mixing with Diffusion Modelscited by 63
  37. 2022Training Language Models with Language Feedbackcited by 48
  38. 2023Tuning computer vision models with task rewardscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 53; 2025: 70; 2026: 25.

172 papers.

Sort:

By area:

  1. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  2. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  3. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  4. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  5. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  6. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  7. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  8. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  9. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  10. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  11. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  12. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
38 more
  1. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 182
  2. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
  3. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  4. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 105
  5. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  6. 2024DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editingcited by 92
  7. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 85
  8. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  9. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  10. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  11. 2023Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulationcited by 61
  12. 2023ZONE: Zero-Shot Instruction-Guided Local Editingcited by 61
  13. 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 61
  14. 2025Se~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialistscited by 61
  15. 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modelingcited by 61
  16. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 59
  17. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  18. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 52
  19. 2024Inference-Time Alignment of Diffusion Models with Direct Noise Optimizationcited by 51
  20. 2024AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policiescited by 50
  21. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 42
  22. 2024PromptFix: You Prompt and We Fix the Photocited by 40
  23. 2024Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directionscited by 39
  24. 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them Firstcited by 39
  25. 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
  26. 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 37
  27. 2023SHAP-EDITOR: Instruction-guided Latent 3D Editing in Secondscited by 36
  28. 2024Full-Atom Peptide Design based on Multi-modal Flow Matchingcited by 34
  29. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 34
  30. 2024Learning Action and Reasoning-Centric Image Editing from Videos and Simulationscited by 32
  31. 2025FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Modelcited by 31
  32. 2025InstructX: Towards Unified Visual Editing with MLLM Guidancecited by 31
  33. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  34. 2024HumanEdit: A High-Quality Human-Rewarded Dataset for Instruction-based Image Editingcited by 29
  35. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  36. 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
  37. 2023EditShield: Protecting Unauthorized Image Editing by Instruction-guided Diffusion Modelscited by 27
  38. 2023LIME: Localized Image Editing via Attention Regularization in Diffusion Modelscited by 27

50 most cited of 172 All 172 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing
  2. 2026HP-Edit: A Human-Preference Post-Training Framework for Image Editing
  3. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
  4. 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing
  5. 2026RewardHarness: Self-Evolving Agentic Post-Training
  6. 2026Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis
  7. 2024Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
  8. 2026Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling