mlsanityData to 7 Oct 2026

Comparative Study of Large Language Models on Chinese Film Script Continuation: An Empirical Analysis Based on GPT-5.2 and Qwen-Max

arXiv 2601.14826 ↗First posted 21 Jan 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) are increasingly applied to creative writing, their performance on culturally specific narrative tasks warrants systematic investigation. This study constructs the first Chinese film script continuation benchmark comprising 53 classic films, and designs a multi-dimensional evaluation framework comparing GPT-5.2 and Qwen-Max-Latest. Using a "first half to second half" continuation paradigm with 3 samples per film, we obtained 303 valid samples (GPT-5.2: 157, 98.7% validity; Qwen-Max: 146, 91.8% validity). Evaluation integrates ROUGE-L, Structural Similarity, and LLM-as-Judge scoring (DeepSeek-Reasoner). Statistical analysis of 144 paired samples reveals: Qwen-Max achieves marginally higher ROUGE-L (0.2230 vs 0.2114, d=-0.43); however, GPT-5.2 significantly outperforms in structural preservation (0.93 vs 0.75, d=0.46), overall quality (44.79 vs 25.72, d=1.04), and composite scores (0.50 vs 0.39, d=0.84). The overall quality effect size reaches large effect level (d>0.8). GPT-5.2 excels in character consistency, tone-style matching, and format preservation, while Qwen-Max shows deficiencies in generation stability. This study provides a reproducible framework for LLM evaluation in Chinese creative writing.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 24 references that are in the map. First posted on arXiv 21 Jan 2026.

Builds on

8 of 24 references.

References · 8

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2018Hierarchical Neural Story Generationcited by 1,182
  6. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  7. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  8. 2020PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Trackingcited by 104

Built on by

Not cited yet.