# Exploring the Utilities of the Rationales from Large Language Models to Enhance Automated Essay Scoring

arXiv: https://arxiv.org/abs/2510.27131 · HTML: https://mlsanity.com/paper/2510.27131/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [Long Document Summarization with Top-down and Bottom-up Inference](https://mlsanity.com/paper/2203.07586/) (2022)
- [Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs](https://mlsanity.com/paper/2410.14202/) (2024)
- [Distilling ChatGPT for Explainable Automated Student Answer Assessment](https://mlsanity.com/paper/2305.12962/) (2023)
- [LCES: Zero-shot Automated Essay Scoring via Pairwise Comparisons Using Large Language Models](https://mlsanity.com/paper/2505.08498/) (2025)
- [Using ChatGPT to Score Essays and Short-Form Constructed Responses](https://mlsanity.com/paper/2408.09540/) (2024)
- [Rubric-Specific Approach to Automated Essay Scoring with Augmentation Training](https://mlsanity.com/paper/2309.02740/) (2023)

## Built on by (most cited first, 1 of 1)

- [Empirical Comparison of Encoder-Based Language Models and Feature-Based Supervised Machine Learning Approaches to Automated Scoring of Long Essays](https://mlsanity.com/paper/2601.02659/) (2026)

## Abstract (arXiv)

This study explored the utilities of rationales generated by GPT-4.1 and GPT-5 in automated scoring using Prompt 6 essays from the 2012 Kaggle ASAP data. Essay-based scoring was compared with rationale-based scoring. The study found in general essay-based scoring performed better than rationale-based scoring with higher Quadratic Weighted Kappa (QWK). However, rationale-based scoring led to higher scoring accuracy in terms of F1 scores for score 0 which had less representation due to class imbalance issues. The ensemble modeling of essay-based scoring models increased the scoring accuracy at both specific score levels and across all score levels. The ensemble modeling of essay-based scoring and each of the rationale-based scoring performed about the same. Further ensemble of essay-based scoring and both rationale-based scoring yielded the best scoring accuracy with QWK of 0.870 compared with 0.848 reported in literature.