# Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity

arXiv: https://arxiv.org/abs/2503.00046 · HTML: https://mlsanity.com/paper/2503.00046/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 13 of 13)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Towards a Unified Multi-Dimensional Evaluator for Text Generation](https://mlsanity.com/paper/2210.07197/) (2022)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks](https://mlsanity.com/paper/2310.00752/) (2023)
- [Automatic Understanding of Image and Video Advertisements](https://mlsanity.com/paper/1707.03067/) (2017)
- [AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text](https://mlsanity.com/paper/2410.04265/) (2024)
- [Benchmarking Language Model Creativity: A Case Study on Code Generation](https://mlsanity.com/paper/2407.09007/) (2024)
- [Probing the Creativity of Large Language Models: Can models produce divergent semantic association?](https://mlsanity.com/paper/2310.11158/) (2023)
- [I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors](https://mlsanity.com/paper/2305.14724/) (2023)
- [Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?](https://mlsanity.com/paper/2407.01119/) (2024)
- [CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints](https://mlsanity.com/paper/2410.04197/) (2024)
- [Every Answer Matters: Evaluating Commonsense with Probabilistic Measures](https://mlsanity.com/paper/2406.04145/) (2024)

## Abstract (arXiv)

Evaluating creativity is challenging, even for humans, not only because of its subjectivity but also because it involves complex cognitive processes. Inspired by work in marketing, we attempt to break down visual advertisement creativity into atypicality and originality. With fine-grained human annotations on these dimensions, we propose a suite of tasks specifically for such a subjective problem. We also evaluate the alignment between state-of-the-art (SoTA) vision language models (VLMs) and humans on our proposed benchmark, demonstrating both the promises and challenges of using VLMs for automatic creativity assessment.