CLIN: an Objective Framework for Evaluating Creativity in Short Persian Literary Text
Not on the map. Why?
Abstract (arXiv)
Evaluating creativity in large language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine how reliably LLMs evaluate short literary text in Persian, a low-resource language, across multiple evaluation strategies and prompt formulations. We find that LLM-human agreement varies substantially across dimensions: alignment is stronger for structured TTCT-derived properties such as Originality, Fluency, and Elaboration, but considerably weaker for more subjective dimensions, particularly Emotion and Attractiveness. Judgments are also sensitive to prompt formulation, while few-shot prompting, ensembling, and multi-agent debate provide no consistent improvement. Motivated by this dimension-dependent behavior, we investigate whether structured creativity dimensions can instead be approximated using simple, interpretable proxy metrics. We introduce CLIN, which evaluates three TTCT-derived dimensions separately using topic-aware novelty for Originality, contextual lexical clustering for Fluency, and lexical diversity for Elaboration. These proxies achieve human alignment comparable to or better than the strongest zero-shot LLM judge in our setting while requiring substantially lower evaluation cost.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
14 of 34 references.
References · 14
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
- 2018Targeted Syntactic Evaluation of Language Modelscited by 239
- 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 168
- 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
- 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writingcited by 39
- 2024Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?cited by 26
- 2025Creative Preference Optimizationcited by 24
- 2025Automated Creativity Evaluation for Large Language Models: A Reference-Based Approachcited by 20
- 2025Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluationscited by 12
- 2025Deep Associations, High Creativity: A Simple yet Effective Metric for Evaluating Large Language Modelscited by 6
- 2023Creative Data Generation: A Review Focusing on Text and Poetrycited by 5
- 2025Evaluating the Creativity of LLMs in Persian Literary Text Generationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
- 2025Curiosity-Driven LLM-as-a-judge for Personalized Creative Judgment
- 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models
- 2025Creativity Benchmark: A benchmark for marketing creativity for large language models
- 2024Creativity Has Left the Chat: The Price of Debiasing Language Models
- 2026QUIET: A Multi-Blank Cascaded Story Cloze Benchmark for LLM Creative Generation Capability
- 2024Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publications
- 2025CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product