mlsanityData to 14 Oct 2026

Rich Human Feedback for Text-to-Image Generation

arXiv 2312.10240 ↗First posted 15 Dec 2023cs.CVcited by 141

On the map: Diffusion models → Image generation, text-to-image and editing

also in 1 more
Abstract (arXiv)

Recent Text-to-Image (T2I) generation models such as Stable Diffusion and Imagen have made significant progress in generating high-resolution images based on text descriptions. However, many generated images still suffer from issues such as artifacts/implausibility, misalignment with text descriptions, and low aesthetic quality. Inspired by the success of Reinforcement Learning with Human Feedback (RLHF) for large language models, prior works collected human-provided scores as feedback on generated images and trained a reward model to improve the T2I generation. In this paper, we enrich the feedback signal by (i) marking image regions that are implausible or misaligned with the text, and (ii) annotating which words in the text prompt are misrepresented or missing on the image. We collect such rich human feedback on 18K generated images (RichHF-18K) and train a multimodal transformer to predict the rich feedback automatically. We show that the predicted rich human feedback can be leveraged to improve image generation, for example, by selecting high-quality training data to finetune and improve the generative models, or by creating masks with predicted heatmaps to inpaint the problematic regions. Notably, the improvements generalize to models (Muse) beyond those used to generate the images on which human feedback data were collected (Stable Diffusion variants). The RichHF-18K data set will be released in our GitHub repository: https://github.com/google-research/google-research/tree/master/richhf_18k.

Key facts. Cited by 141 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 53 of its 63 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

53 of 63 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Text-to-image Diffusion Models in Generative AI: A Survey14 Mar
  2. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting13 Dec
  3. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  4. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  11. 2017Neural Discrete Representation Learningcited by 4,712
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
35 more
  1. 2013Auto-Encoding Variational Bayescited by 4,053
  2. 2016Improved Techniques for Training GANscited by 2,970
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  4. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  8. 2023PaLM 2 Technical Reportcited by 1,162
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  10. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  11. 2022Diffusion Models in Vision: A Surveycited by 1,015
  12. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  13. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  14. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  15. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  16. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  17. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  18. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  19. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  20. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  21. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  22. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 260
  23. 2023A Survey on Video Diffusion Modelscited by 206
  24. 2019Controllable Text-to-Image Generationcited by 187
  25. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  26. 2016What do different evaluation metrics tell us about saliency models?cited by 185
  27. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 181
  28. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  29. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 171
  30. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 162
  31. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  32. 2023Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generationcited by 68
  33. 2022Spotlight: Mobile UI Understanding using Vision-Language Models with a Focuscited by 62
  34. 2023Visual Programming for Text-to-Image Generation and Evaluationcited by 54
  35. 2023Perceptual Artifacts Localization for Image Synthesis Taskscited by 43

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 33; 2025: 74; 2026: 34.

141 papers.

Sort:

By area:

  1. 2025Improving Video Generation with Human Feedbackcited by 284
  2. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 199
  3. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 156
  4. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  5. 2025LEGION: Learning to Ground and Explain for Synthetic Image Detectioncited by 65
  6. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 62
  7. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 53
  8. 2025Text-to-CAD Generation Through Infusing Visual Feedback in Large Language Modelscited by 47
  9. 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
  10. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 39
  11. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 34
  12. 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generationcited by 33
38 more
  1. 2024Scalable Ranked Preference Optimization for Text-to-Image Generationcited by 33
  2. 2024AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMMcited by 24
  3. 2025Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Modelscited by 23
  4. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Imagecited by 20
  5. 2025HumanAesExpert: Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessmentcited by 19
  6. 2024A Survey on Quality Metrics for Text-to-Image Generationcited by 18
  7. 2024VersaT2I: Improving Text-to-Image Models with Versatile Rewardcited by 17
  8. 2024Aligning Human Motion Generation with Human Perceptionscited by 16
  9. 2024Towards Reliable Advertising Image Generation Using Human Feedbackcited by 16
  10. 2025Enhancing Intent Understanding for Ambiguous prompt: A Human-Machine Co-Adaption Strategycited by 16
  11. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 15
  12. 2025Towards Transformer-Based Aligned Generation with Self-Coherence Guidancecited by 14
  13. 2024HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluatorcited by 13
  14. 2025AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Imagescited by 13
  15. 2025Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferencescited by 13
  16. 2024Aligning Few-Step Diffusion Models with Dense Reward Difference Learningcited by 12
  17. 2024Detecting Human Artifacts from Text-to-Image Modelscited by 12
  18. 2024Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-bodycited by 12
  19. 2025Erasing More Than Intended? How Concept Erasure Degrades the Generation of Non-Target Conceptscited by 12
  20. 2025ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modelingcited by 12
  21. 2024Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learningcited by 11
  22. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11
  23. 2024F-Bench: Rethinking Human Preference Evaluation Metrics for Benchmarking Face Generation, Customization, and Restorationcited by 10
  24. 2025EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editingcited by 10
  25. 2026Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuningcited by 10
  26. 2024Bridging SFT and DPO for Diffusion Model Alignment with Self-Sampling Preference Optimizationcited by 9
  27. 2025Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generationcited by 9
  28. 2025VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Controlcited by 9
  29. 2025Context-guided Responsible Data Augmentation with Diffusion Modelscited by 9
  30. 2025Aligning Anime Video Generation with Human Feedbackcited by 9
  31. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  32. 2025MHR: Momentum Human Rigcited by 9
  33. 2025PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewardscited by 9
  34. 2024Preference Adaptive and Sequential Text-to-Image Generationcited by 8
  35. 2025NTIRE 2025 challenge on Text to Image Generation Model Quality Assessmentcited by 8
  36. 2024What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Modelscited by 7
  37. 2024CoMPaSS: Enhancing Spatial Understanding in Text-to-Image Diffusion Modelscited by 7
  38. 2025DiffDoctor: Diagnosing Image Diffusion Models Before Treatingcited by 7

50 most cited of 141 All 141 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Aligning Text-to-Image Models using Human Feedback
  2. 2024Enhancing Image Caption Generation Using Reinforcement Learning with Human Feedback
  3. 2025Test-time Prompt Refinement for Text-to-Image Models
  4. 2025CHATS: Combining Human-Aligned Optimization and Test-Time Sampling for Text-to-Image Generation
  5. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  6. 2024Dynamic Prompt Optimizing for Text-to-Image Generation
  7. 2024Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models
  8. 2026Unified Text-Image Generation with Weakness-Targeted Post-Training