mlsanityData to 7 Oct 2026

Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation

arXiv 2305.01569 ↗First posted 2 May 2023cs.CV cs.AIcited by 950

Not on the map. Why?

Cited by 468 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The ability to collect a large dataset of human preferences from text-to-image users is usually limited to companies, making such datasets inaccessible to the public. To address this issue, we create a web app that enables text-to-image users to generate images and specify their preferences. Using this web app we build Pick-a-Pic, a large, open dataset of text-to-image prompts and real users' preferences over generated images. We leverage this dataset to train a CLIP-based scoring function, PickScore, which exhibits superhuman performance on the task of predicting human preferences. Then, we test PickScore's ability to perform model evaluation and observe that it correlates better with human rankings than other automatic evaluation metrics. Therefore, we recommend using PickScore for evaluating future text-to-image generation models, and using Pick-a-Pic prompts as a more relevant dataset than MS-COCO. Finally, we demonstrate how PickScore can enhance existing text-to-image models via ranking.

Key facts. Cited by 950 papers in the map (data through 7 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

13 of 20 references.

References · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  11. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  12. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
1 more
  1. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 62; 2024: 189; 2025: 351; 2026: 348.

950 papers.

Sort:

By area:

  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  3. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  4. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  5. 2023Adversarial Diffusion Distillationcited by 741
  6. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  7. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  8. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  9. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  10. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  11. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  12. 2025Improving Video Generation with Human Feedbackcited by 267
38 more
  1. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  2. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  3. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  4. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
  5. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  6. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  7. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 172
  8. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
  9. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  10. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  11. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedbackcited by 153
  12. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  13. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 148
  14. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  15. 2023Rich Human Feedback for Text-to-Image Generationcited by 139
  16. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
  17. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 116
  18. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  19. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 111
  20. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  21. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  22. 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 102
  23. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  24. 2024Video Diffusion Alignment via Reward Gradientscited by 101
  25. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  26. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 99
  27. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  28. 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 91
  29. 2025Test-time Alignment of Diffusion Models without Reward Over-optimizationcited by 89
  30. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 88
  31. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learningcited by 88
  32. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  33. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  34. 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generationcited by 85
  35. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 84
  36. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  37. 2024Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimizationcited by 81
  38. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 81

50 most cited of 950 All 950 (JSON)