mlsanityData to 7 Oct 2026

Human Preference Score: Better Aligning Text-to-Image Models with Human Preference

arXiv 2303.14420 ↗First posted 25 Mar 2023cs.CV cs.AIcited by 284

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Recent years have witnessed a rapid growth of deep generative models, with text-to-image models gaining significant attention from the public. However, existing models often generate images that do not align well with human preferences, such as awkward combinations of limbs and facial expressions. To address this issue, we collect a dataset of human choices on generated images from the Stable Foundation Discord channel. Our experiments demonstrate that current evaluation metrics for generative models do not correlate well with human choices. Thus, we train a human preference classifier with the collected dataset and derive a Human Preference Score (HPS) based on the classifier. Using HPS, we propose a simple yet effective method to adapt Stable Diffusion to better align with human preferences. Our experiments show that HPS outperforms CLIP in predicting human choices and has good generalization capability toward images generated from other models. By tuning Stable Diffusion with the guidance of HPS, the adapted model is able to generate images that are more preferred by human users. The project page is available here: https://tgxs002.github.io/align_sd_web/ .

Key facts. Cited by 284 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 40 of its 48 references that are in the map. First posted on arXiv 25 Mar 2023.

Builds on

40 of 48 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
28 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2016Improved Techniques for Training GANscited by 2,938
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  10. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  11. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  12. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  13. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  14. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  15. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  16. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  17. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  18. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
  19. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  20. 2018A Note on the Inception Scorecited by 225
  21. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  22. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  23. 2015Generating Images from Captions with Attentioncited by 135
  24. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  25. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Expertscited by 110
  26. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  27. 2021ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generationcited by 41
  28. 2018DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedbackcited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 87; 2025: 113; 2026: 71.

284 papers.

Sort:

By area:

  1. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  2. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  3. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  4. 2025Improving Video Generation with Human Feedbackcited by 267
  5. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  6. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  7. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  8. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  9. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  10. 2023Rich Human Feedback for Text-to-Image Generationcited by 139
  11. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 116
  12. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 99
38 more
  1. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  2. 2025Test-time Alignment of Diffusion Models without Reward Over-optimizationcited by 89
  3. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  4. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 84
  5. 2024Large-scale Reinforcement Learning for Diffusion Modelscited by 78
  6. 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 75
  7. 2024Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimizationcited by 75
  8. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 71
  9. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 63
  10. 2025RewardDance: Reward Scaling in Visual Generationcited by 63
  11. 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modelingcited by 61
  12. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  13. 2024Deep Reward Supervisions for Tuning Text-to-Image Diffusion Modelscited by 59
  14. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 59
  15. 2024A Dense Reward View on Aligning Text-to-Image Diffusion with Preferencecited by 55
  16. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  17. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 50
  18. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 45
  19. 2025Distribution Matching Distillation Meets Reinforcement Learningcited by 43
  20. 2024PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Modelscited by 41
  21. 2025Calibrated Multi-Preference Optimization for Aligning Diffusion Modelscited by 40
  22. 2024Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessmentcited by 39
  23. 2024Autoregressive Models in Vision: A Surveycited by 39
  24. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 38
  25. 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewritingcited by 38
  26. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcingcited by 37
  27. 2024Curriculum Direct Preference Optimization for Diffusion and Consistency Modelscited by 36
  28. 2024Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generationcited by 35
  29. 2024Reward Guided Latent Consistency Distillationcited by 35
  30. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  31. 2026DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Modelscited by 35
  32. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 34
  33. 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
  34. 2025NoHumansRequired: Autonomous High-Quality Image Editing Triplet Miningcited by 33
  35. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 32
  36. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  37. 2024CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibrationcited by 30
  38. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30

50 most cited of 284 All 284 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
  2. 2025PC-Diffusion: Aligning Diffusion Models with Human Preferences via Preference Classifier
  3. 2023Diffusion Model Alignment Using Direct Preference Optimization
  4. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
  5. 2026Can We Predict The Human Preference For Text-to-Image Content Prior To Generation And Is It Even Useful To Do So?
  6. 2024Scalable Ranked Preference Optimization for Text-to-Image Generation
  7. 2025BalancedDPO: Adaptive Multi-Metric Alignment
  8. 2025Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking