# Evaluating the Creativity of LLMs in Persian Literary Text Generation

arXiv: https://arxiv.org/abs/2509.18401 · HTML: https://mlsanity.com/paper/2509.18401/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 9 of 9)

- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing](https://mlsanity.com/paper/2310.08433/) (2023)
- [Are Large Language Models Capable of Generating Human-Level Narratives?](https://mlsanity.com/paper/2407.13248/) (2024)
- [Putting GPT-3's Creativity to the (Alternative Uses) Test](https://mlsanity.com/paper/2206.08932/) (2022)
- [A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians](https://mlsanity.com/paper/2405.20956/) (2024)
- [CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis](https://mlsanity.com/paper/2406.12665/) (2024)
- [CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints](https://mlsanity.com/paper/2410.04197/) (2024)
- [Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset](https://mlsanity.com/paper/2306.11167/) (2023)

## Built on by (most cited first, 4 of 4)

- [GhazalBench: Canonical Verse Access in LLMs across Persian Ghazals and Shakespearean Sonnets](https://mlsanity.com/paper/2603.09979/) (2026)
- [PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions](https://mlsanity.com/paper/2605.27015/) (2026)
- [CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations](https://mlsanity.com/paper/2607.15847/) (2026)
- [CLIN: an Objective Framework for Evaluating Creativity in Short Persian Literary Text](https://mlsanity.com/paper/2608.30754/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated notable creative abilities in generating literary texts, including poetry and short stories. However, prior research has primarily centered on English, with limited exploration of non-English literary traditions and without standardized methods for assessing creativity. In this paper, we evaluate the capacity of LLMs to generate Persian literary text enriched with culturally relevant expressions. We build a dataset of user-generated Persian literary spanning 20 diverse topics and assess model outputs along four creativity dimensions-originality, fluency, flexibility, and elaboration-by adapting the Torrance Tests of Creative Thinking. To reduce evaluation costs, we adopt an LLM as a judge for automated scoring and validate its reliability against human judgments using intraclass correlation coefficients, observing strong agreement. In addition, we analyze the models' ability to understand and employ four core literary devices: simile, metaphor, hyperbole, and antithesis. Our results highlight both the strengths and limitations of LLMs in Persian literary text generation, underscoring the need for further refinement.