mlsanityData to 7 Oct 2026

Automatic Assessment of Divergent Thinking in Chinese Language with TransDis: A Transformer-Based Language Model Approach

arXiv 2306.14790 ↗First posted 26 Jun 2023cs.CL stat.APcited by 2

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Language models have been increasingly popular for automatic creativity assessment, generating semantic distances to objectively measure the quality of creative ideas. However, there is currently a lack of an automatic assessment system for evaluating creative ideas in the Chinese language. To address this gap, we developed TransDis, a scoring system using transformer-based language models, capable of providing valid originality (quality) and flexibility (variety) scores for Alternative Uses Task (AUT) responses in Chinese. Study 1 demonstrated that the latent model-rated originality factor, comprised of three transformer-based models, strongly predicted human originality ratings, and the model-rated flexibility strongly correlated with human flexibility ratings as well. Criterion validity analyses indicated that model-rated originality and flexibility positively correlated to other creativity measures, demonstrating similar validity to human ratings. Study 2 & 3 showed that TransDis effectively distinguished participants instructed to provide creative vs. common uses (Study 2) and participants instructed to generate ideas in a flexible vs. persistent way (Study 3). Our findings suggest that TransDis can be a reliable and low-cost tool for measuring idea originality and flexibility in Chinese language, potentially paving the way for automatic creativity assessment in other languages. We offer an open platform to compute originality and flexibility for AUT responses in Chinese and over 50 other languages (https://osf.io/59jv2/).

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 107 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

14 of 107 references.

References · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  7. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  11. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  12. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
2 more
  1. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  2. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experimentcited by 22
  2. 2024Do LLMs Agree on the Creativity Evaluation of Alternative Uses?cited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2025S-DAT: A Multilingual, GenAI-Driven Framework for Automated Divergent Thinking Assessment
  2. 2025CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
  3. 2025COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processes
  4. 2023Fluid Transformers and Creative Analogies: Exploring Large Language Models' Capacity for Augmenting Cross-Domain Analogical Creativity
  5. 2026Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
  6. 2025Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations
  7. 2026How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning
  8. 2025Curiosity-Driven LLM-as-a-judge for Personalized Creative Judgment