Personalized Reward Modeling for Text-to-Image Generation
On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
Abstract (arXiv)
Recent text-to-image (T2I) models generate semantically coherent images from textual prompts, yet evaluating how well they align with individual user preferences remains an open challenge. Conventional evaluation methods, general reward functions or similarity-based metrics, fail to capture the diversity and complexity of personal visual tastes. In this work, we present PIGReward, a personalized reward model that dynamically generates user-conditioned evaluation dimensions and assesses images through CoT reasoning. To address the scarcity of user data, PIGReward adopt a self-bootstrapping strategy that reasons over limited reference data to construct rich user contexts, enabling personalization without user-specific training. Beyond evaluation, PIGReward provides personalized feedback that drives user-specific prompt optimization, improving alignment between generated images and individual intent. We further introduce PIGBench, a per-user preference benchmark capturing diverse visual interpretations of shared prompts. Extensive experiments demonstrate that PIGReward surpasses existing methods in both accuracy and interpretability, establishing a scalable and reasoning-based foundation for personalized T2I evaluation and optimization. Taken together, our findings highlight PIGReward as a robust steptoward individually aligned T2I generation.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 64 of its 76 references that are in the map. First posted on arXiv 21 Nov 2025.
Builds on
64 of 76 references.
Closest ancestors · 10
Same direction, newest first
- 2025Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization6 Nov
- 2025Multimodal LLMs as Customized Reward Models for Text-to-Image Generation28 Jul
- 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning6 May
- 2025Unified Reward Model for Multimodal Understanding and Generation7 Mar
- 2025Personalized Preference Fine-tuning of Diffusion Models11 Jan
- 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation30 Dec
- 2024VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment12 Oct
- 2023Rich Human Feedback for Text-to-Image Generation15 Dec
- 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
- 2023FABRIC: Personalizing Diffusion Models with Iterative Feedback19 Jul
Other references · 54
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
38 more
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2016Improved Techniques for Training GANscited by 2,938
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
- 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
- 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 395
- 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
- 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
- 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
- 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
- 2024A Roadmap to Pluralistic Alignmentcited by 175
- 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 147
- 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 135
- 2024LLaVA-Critic: Learning to Evaluate Multimodal Modelscited by 132
- 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 102
- 2024Calibrated Self-Rewarding Vision Language Modelscited by 77
- 2023Group Preference Optimization: Few-Shot Alignment of Large Language Modelscited by 61
- 2024PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferencescited by 53
- 2025Evaluating the Diversity and Quality of LLM Generated Contentcited by 51
- 2025Personalized Generation In Large Model Era: A Surveycited by 45
- 2025SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMscited by 26
- 2024Self-rationalization improves LLM as a fine-grained judgecited by 25
- 2023Prompt Expansion for Adaptive Text-to-Image Generationcited by 24
- 2023Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewritingcited by 23
- 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
- 2024PMG : Personalized Multimodal Generation with Large Language Modelscited by 19
- 2024Guided Profile Generation Improves Personalization with LLMscited by 19
- 2024ViPer: Visual Personalization of Generative Models via Individual Preference Learningcited by 17
- 2024Personalized Image Generation with Large Multimodal Modelscited by 14
- 2024Self-Consistent Reasoning-based Aspect-Sentiment Quad Prediction with Extract-Then-Assign Strategycited by 12
- 2024Preference Adaptive and Sequential Text-to-Image Generationcited by 8
- 2025Imagine All The Relevance: Scenario-Profiled Indexing with Knowledge Expansion for Dense Retrievalcited by 5
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
- 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting
- 2026DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation
- 2026Personalize at Test Time: Learning User Preferences for Image Generation
- 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
- 2025Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models
- 2026Unified Personalized Reward Model for Vision Generation
- 2024Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation