# Fine-tuning ChatGPT for Automatic Scoring of Written Scientific Explanations in Chinese

arXiv: https://arxiv.org/abs/2501.06704 · HTML: https://mlsanity.com/paper/2501.06704/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 56 references that are in the map.
- First posted on arXiv 12 Jan 2025.

## Builds on (most cited first, 7 of 7)

- [Fine-tuning ChatGPT for Automatic Scoring](https://mlsanity.com/paper/2310.10072/) (2023)
- [Context Matters: A Strategy to Pre-train Language Model for Science Education](https://mlsanity.com/paper/2301.12031/) (2023)
- [Is Word Segmentation Necessary for Deep Learning of Chinese Representations?](https://mlsanity.com/paper/1905.05526/) (2019)
- [Redefining Qualitative Analysis in the AI Era: Utilizing ChatGPT for Efficient Thematic Analysis](https://mlsanity.com/paper/2309.10771/) (2023)
- [Subword Encoding in Lattice LSTM for Chinese Word Segmentation](https://mlsanity.com/paper/1810.12594/) (2018)
- [AI and Machine Learning for Next Generation Science Assessments](https://mlsanity.com/paper/2405.06660/) (2024)
- [EvoGrader: an online formative assessment tool for automatically evaluating written evolutionary explanations](https://mlsanity.com/paper/1601.03348/) (2016)

## Built on by (most cited first, 3 of 3)

- [Evolution of AI in Education: Agentic Workflows](https://mlsanity.com/paper/2504.20082/) (2025)
- [Comparing Human and AI Rater Effects Using the Many-Facet Rasch Model](https://mlsanity.com/paper/2505.18486/) (2025)
- [Using Large Language Models to Assess Teachers' Pedagogical Content Knowledge](https://mlsanity.com/paper/2505.19266/) (2025)

## Abstract (arXiv)

The development of explanations for scientific phenomena is essential in science assessment, but scoring student-written explanations remains challenging and resource-intensive. Large language models (LLMs) have shown promise in addressing this issue, particularly in alphabetic languages like English. However, their applicability to logographic languages is less explored. This study investigates the potential of fine-tuning ChatGPT, a leading LLM, to automatically score scientific explanations written in Chinese. Student responses to seven scientific explanation tasks were collected and automatically scored, with scoring accuracy examined in relation to reasoning complexity using the Kendall correlation. A qualitative analysis explored how linguistic features influenced scoring accuracy. The results show that domain-specific adaptation enables ChatGPT to score Chinese scientific explanations with accuracy. However, scoring accuracy correlates with reasoning complexity: a negative correlation for lower-level responses and a positive one for higher-level responses. The model overrates complex reasoning in low-level responses with intricate sentence structures and underrates high-level responses using concise causal reasoning. These correlations stem from linguistic features--simplicity and clarity enhance accuracy for lower-level responses, while comprehensiveness improves accuracy for higher-level ones. Simpler, shorter responses tend to score more accurately at lower levels, whereas longer, information-rich responses yield better accuracy at higher levels. These findings demonstrate the effectiveness of LLMs in automatic scoring within a Chinese context and emphasize the importance of linguistic features and reasoning complexity in fine-tuning scoring models for educational assessments.