mlsanityData to 7 Oct 2026

Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

arXiv 2306.09341 ↗First posted 15 Jun 2023cs.CV cs.AI cs.DBcited by 847

Not on the map. Why?

Cited by 427 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent text-to-image generative models can generate high-fidelity images from text inputs, but the quality of these generated images cannot be accurately evaluated by existing evaluation metrics. To address this issue, we introduce Human Preference Dataset v2 (HPD v2), a large-scale dataset that captures human preferences on images from a wide range of sources. HPD v2 comprises 798,090 human preference choices on 433,760 pairs of images, making it the largest dataset of its kind. The text prompts and images are deliberately collected to eliminate potential bias, which is a common issue in previous datasets. By fine-tuning CLIP on HPD v2, we obtain Human Preference Score v2 (HPS v2), a scoring model that can more accurately predict human preferences on generated images. Our experiments demonstrate that HPS v2 generalizes better than previous metrics across various image distributions and is responsive to algorithmic improvements of text-to-image generative models, making it a preferable evaluation metric for these models. We also investigate the design of the evaluation prompts for text-to-image generative models, to make the evaluation stable, fair and easy-to-use. Finally, we establish a benchmark for text-to-image generative models using HPS v2, which includes a set of recent text-to-image models from the academic, community and industry. The code and dataset is available at https://github.com/tgxs002/HPSv2 .

Key facts. Cited by 847 papers in the map (data through 7 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

30 of 53 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
18 more
  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  3. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  4. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  5. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  6. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  7. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  8. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
  9. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  10. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  11. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  12. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  13. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 126
  14. 2021FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizationcited by 71
  15. 2023Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Imagescited by 65
  16. 2019HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Modelscited by 59
  17. 2023A Perceptual Quality Assessment Exploration for AIGC Imagescited by 54
  18. 2022Proper Reuse of Image Classification Features Improves Object Detectioncited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 154; 2025: 343; 2026: 330.

847 papers.

Sort:

By area:

  1. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  2. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  3. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  4. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  5. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  6. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  7. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  8. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  9. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  10. 2025Improving Video Generation with Human Feedbackcited by 267
  11. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  12. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 222
38 more
  1. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  2. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
  3. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 200
  4. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  5. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  6. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  7. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  8. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
  9. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  10. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedbackcited by 153
  11. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 148
  12. 2025Seedream 3.0 Technical Reportcited by 146
  13. 2023Rich Human Feedback for Text-to-Image Generationcited by 139
  14. 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
  15. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
  16. 2023GPT-4V(ision) as a Generalist Evaluator for Vision-Language Taskscited by 118
  17. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  18. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  19. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  20. 2023SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillationcited by 109
  21. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  22. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  23. 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 102
  24. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  25. 2024Video Diffusion Alignment via Reward Gradientscited by 101
  26. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 99
  27. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  28. 2025Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generationcited by 95
  29. 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Modelscited by 93
  30. 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 91
  31. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 88
  32. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learningcited by 88
  33. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  34. 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generationcited by 85
  35. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 85
  36. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 84
  37. 2024Phased Consistency Modelscited by 83
  38. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82

50 most cited of 847 All 847 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
  2. 2026BAFIS: Dataset + Framework to assess occupational Bias and Human Preference in modern Text-to-image Models
  3. 2026Personalize at Test Time: Learning User Preferences for Image Generation
  4. 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  5. 2026Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation
  6. 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset
  7. 2025Instant Preference Alignment for Text-to-Image Diffusion Models
  8. 2026K-Sort Eval: Efficient Preference Evaluation for Visual Generation via Corrected VLM-as-a-Judge