COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processes
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large language models exhibit systematic deficiencies in creative writing, particularly in non-English contexts where training data is scarce and lacks process-level supervision. We present COIG-Writer, a novel Chinese creative writing dataset that captures both diverse outputs and their underlying thought processes through systematic reverse-engineering of high-quality texts. Unlike existing datasets that provide only input-output pairs, COIG-Writer comprises 1,665 meticulously curated triplets spanning 51 genres, each containing: (1) a reverse-engineered prompt, (2) detailed creative reasoning documenting decision-making processes, and (3) the final text. Through comprehensive experiments, we identify a two-component model of creative writing: narrative logic (provided by process supervision) and linguistic expression (maintained by general-purpose data). Our findings reveal three critical insights: (1) Process supervision is highly effective but requires stabilization with general data. A ratio of at least one creative sample to twelve general samples is needed to achieve optimal performance; below this threshold, the win rate progressively degrades (from 62.75% down to 35.78%)., (2) creative capabilities are culturally-bound with no cross-lingual transfer (89.26pp gap between Chinese and English performance), and (3) lexical diversity inversely correlates with creative quality (TTR paradox), suggesting high diversity signals compensatory behavior for logical deficiencies. These findings establish that creative excellence emerges from the interaction between logical scaffolding and linguistic grounding, analogous to how mathematical reasoning enhances but cannot replace linguistic competence in foundation models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 16 of its 36 references that are in the map. First posted on arXiv 16 Oct 2025.
Builds on
16 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2025OpenThoughts: Data Recipes for Reasoning Models4 Jun
- 2025Qwen3 Technical Report14 May
Other references · 14
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2018Hierarchical Neural Story Generationcited by 1,193
- 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
- 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 261
- 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 106
- 2020Content Planning for Neural Story Generation with Aristotelian Rescoringcited by 92
- 2015LCSTS: A Large Scale Chinese Short Text Summarization Datasetcited by 65
- 2024COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuningcited by 45
- 2025LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writingcited by 42
- 2020RoFT: A Tool for Evaluating Human Detection of Machine-Generated Textcited by 36
- 2023Chinese Open Instruction Generalist: A Preliminary Releasecited by 35
- 2024Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Modelcited by 25
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
- 2023Automatic Assessment of Divergent Thinking in Chinese Language with TransDis: A Transformer-Based Language Model Approach
- 2023In BLOOM: Creativity and Affinity in Artificial Lyrics and Art
- 2025Beyond Correctness: Evaluating Subjective Writing Preferences Across Cultures
- 2026Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
- 2026QUIET: A Multi-Blank Cascaded Story Cloze Benchmark for LLM Creative Generation Capability
- 2026When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation
- 2025Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards