mlsanityData to 14 Oct 2026

Can We Predict The Human Preference For Text-to-Image Content Prior To Generation And Is It Even Useful To Do So?

arXiv 2606.05478 ↗First posted 3 Jun 2026cs.CV cs.LGcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Diffusion Models (DM) have revolutionized text-driven generation by enabling the synthesis of high-quality, photorealistic visual content from user prompts. Whereas prior advances in visual generation such as VAEs and GANs were primarily evaluated on perceptual or visual similarity metrics such as FID PSNR, DM advances have fostered the development of more advanced Human Preference Metrics (HPM) that model and quantify human judgment as scalar values. However, DMs synthesize content using an inherently stochastic process where random noise seeds generation. The initial random noise directly affects the quality of generated outputs, both qualitatively and quantitatively. This influence is pronounced in smaller models for local deployment scenarios. Given this phenomenon, we first investigate to what extent we can predict scalar HPM scores prior to committing compute resources for generation. Further, we then investigate to what extent we can leverage such prediction to improve the quality of generated images, and also study which HPMs are best suited for this task. Our investigation reveals that not only is this possible, but that it is feasible to achieve negligible hardware overhead.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 55 of its 74 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

55 of 74 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov

Other references · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
38 more
  1. 2022Video Diffusion Modelscited by 2,305
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  5. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  7. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  8. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  9. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  10. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  11. 2019Panoptic Feature Pyramid Networkscited by 645
  12. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  13. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 394
  14. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  15. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  16. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  17. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
  18. 2020Fast Differentiable Sorting and Rankingcited by 197
  19. 2024A Survey on Diffusion Models for Inverse Problemscited by 180
  20. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimizationcited by 123
  21. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 119
  22. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  23. 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 106
  24. 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 92
  25. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  26. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 87
  27. 2024Diffusion-Based Neural Network Weights Generationcited by 39
  28. 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
  29. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 34
  30. 2025Diffusion Models for Molecules: A Survey of Methods and Taskscited by 19
  31. 2025Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative modelscited by 17
  32. 2024The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generationcited by 14
  33. 2025Source-Guided Flow Matchingcited by 13
  34. 2025Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshapecited by 10
  35. 2025Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidencecited by 8
  36. 2025NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Modelscited by 8
  37. 2025ESS-Flow: Training-free guidance of flow-based models as inference in source spacecited by 8
  38. 2024Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Modelscited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Model-Agnostic Human Preference Inversion in Diffusion Models
  2. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
  3. 2026Na"ive PAINE: Lightweight Text-to-Image Generation Improvement with Prompt Evaluation
  4. 2025When Preferences Diverge: Aligning Diffusion Models with Minority-Aware Adaptive DPO
  5. 2025Instant Preference Alignment for Text-to-Image Diffusion Models
  6. 2024Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models
  7. 2024Aligning Diffusion Models with Noise-Conditioned Perception
  8. 2024Aligning Diffusion Models by Optimizing Human Utility