mlsanityData to 14 Oct 2026

Exploring the Utilities of the Rationales from Large Language Models to Enhance Automated Essay Scoring

arXiv 2510.27131 ↗First posted 31 Oct 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

This study explored the utilities of rationales generated by GPT-4.1 and GPT-5 in automated scoring using Prompt 6 essays from the 2012 Kaggle ASAP data. Essay-based scoring was compared with rationale-based scoring. The study found in general essay-based scoring performed better than rationale-based scoring with higher Quadratic Weighted Kappa (QWK). However, rationale-based scoring led to higher scoring accuracy in terms of F1 scores for score 0 which had less representation due to class imbalance issues. The ensemble modeling of essay-based scoring models increased the scoring accuracy at both specific score levels and across all score levels. The ensemble modeling of essay-based scoring and each of the rationale-based scoring performed about the same. Further ensemble of essay-based scoring and both rationale-based scoring yielded the best scoring accuracy with QWK of 0.870 compared with 0.848 reported in literature.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 11 of its 51 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

11 of 51 references.

References · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  5. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  6. 2022Long Document Summarization with Top-down and Bottom-up Inferencecited by 42
  7. 2024Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMscited by 22
  8. 2023Distilling ChatGPT for Explainable Automated Student Answer Assessmentcited by 14
  9. 2025LCES: Zero-shot Automated Essay Scoring via Pairwise Comparisons Using Large Language Modelscited by 8
  10. 2024Using ChatGPT to Score Essays and Short-Form Constructed Responsescited by 2
  11. 2023Rubric-Specific Approach to Automated Essay Scoring with Augmentation Trainingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Empirical Comparison of Encoder-Based Language Models and Feature-Based Supervised Machine Learning Approaches to Automated Scoring of Long Essayscited by 0