# CREward: A Type-Specific Creativity Reward Model

arXiv: https://arxiv.org/abs/2511.19995 · HTML: https://mlsanity.com/paper/2511.19995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding](https://mlsanity.com/paper/2405.08748/) (2024)
- [Unified Reward Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2503.05236/) (2025)
- [VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation](https://mlsanity.com/paper/2412.21059/) (2024)
- [Rarity Score : A New Metric to Evaluate the Uncommonness of Synthesized Images](https://mlsanity.com/paper/2206.08549/) (2022)
- [SliderSpace: Decomposing the Visual Capabilities of Diffusion Models](https://mlsanity.com/paper/2502.01639/) (2025)
- [ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints](https://mlsanity.com/paper/2308.02669/) (2023)
- [Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework](https://mlsanity.com/paper/2410.21061/) (2024)
- [CREA: A Collaborative Multi-Agent Framework for Creative Image Editing and Generation](https://mlsanity.com/paper/2504.05306/) (2025)
- [Enhancing Creative Generation on Stable Diffusion-based Models](https://mlsanity.com/paper/2503.23538/) (2025)
- [VLM-Eval: A General Evaluation on Video Large Language Models](https://mlsanity.com/paper/2311.11865/) (2023)
- [Creative Blends of Visual Concepts](https://mlsanity.com/paper/2502.16062/) (2025)

## Abstract (arXiv)

Creativity is a complex phenomenon. When it comes to representing and assessing creativity, treating it as a single undifferentiated quantity would appear naive and underwhelming. In this work, we learn the first type-specific creativity reward model, coined CREward, which spans three creativity ``axes," geometry, material, and texture, to allow us to view creativity through the lens of the image formation pipeline. To build our reward model, we first conduct a human benchmark evaluation to capture human perception of creativity for each type across various creative images. We then analyze the correlation between human judgments and predictions by large vision-language models (LVLMs), confirming that LVLMs exhibit strong alignment with human perception. Building on this observation, we collect LVLM-generated labels to train our CREward model that is applicable to both evaluation and generation of creative images. We explore three applications of CREward: creativity assessment, explainable creativity, and creative sample acquisition for both human design inspiration and guiding creative generation through low-rank adaptation.