mlsanityData to 7 Oct 2026

ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

arXiv 2304.05977 ↗First posted 12 Apr 2023cs.CV cs.LGcited by 1,010

Not on the map. Why?

Cited by 511 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a comprehensive solution to learn and improve text-to-image models from human preference feedback. To begin with, we build ImageReward -- the first general-purpose text-to-image human preference reward model -- to effectively encode human preferences. Its training is based on our systematic annotation pipeline including rating and ranking, which collects 137k expert comparisons to date. In human evaluation, ImageReward outperforms existing scoring models and metrics, making it a promising automatic metric for evaluating text-to-image synthesis. On top of it, we propose Reward Feedback Learning (ReFL), a direct tuning algorithm to optimize diffusion models against a scorer. Both automatic and human evaluation support ReFL's advantages over compared methods. All code and datasets are provided at urlhttps://github.com/THUDM/ImageReward.

Key facts. Cited by 1,010 papers in the map (data through 7 Oct 2026). Builds on 61 of its 70 references that are in the map. First posted on arXiv 12 Apr 2023.

Builds on

61 of 70 references.

References · 61

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2017Proximal Policy Optimization Algorithmscited by 14,441
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
38 more
  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  13. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  14. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  15. 2020Learning to summarize from human feedbackcited by 2,562
  16. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  17. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  18. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  19. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  20. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  21. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  22. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  23. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  24. 2021GPT Understands, Toocited by 715
  25. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  26. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  27. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  28. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  29. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  30. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  31. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  32. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  33. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
  34. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  35. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  36. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  37. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  38. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 50; 2024: 176; 2025: 407; 2026: 377.

1,010 papers.

Sort:

By area:

  1. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  2. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  3. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  4. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  5. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  6. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  7. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  8. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  9. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  10. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  11. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  12. 2025Improving Video Generation with Human Feedbackcited by 267
38 more
  1. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  2. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  3. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 246
  4. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  5. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  6. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  7. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  8. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  9. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  10. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 197
  11. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  12. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  13. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  14. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  15. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
  16. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  17. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  18. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  19. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  20. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 158
  21. 2024Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesiscited by 153
  22. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 148
  23. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 143
  24. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  25. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Modelscited by 138
  26. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
  27. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  28. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
  29. 2025From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeerscited by 124
  30. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 116
  31. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  32. 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 113
  33. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 113
  34. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  35. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  36. 2024Video Diffusion Alignment via Reward Gradientscited by 101
  37. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 99
  38. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91

50 most cited of 1,010 All 1,010 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  2. 2025Personalized Reward Modeling for Text-to-Image Generation
  3. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
  4. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences
  5. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  6. 2024DreamReward: Text-to-3D Generation with Human Preference
  7. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editing
  8. 2023Rich Human Feedback for Text-to-Image Generation