# Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs

arXiv: https://arxiv.org/abs/2410.14202 · HTML: https://mlsanity.com/paper/2410.14202/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation](https://mlsanity.com/paper/2205.03835/) (2022)
- [Automated essay scoring with string kernels and word embeddings](https://mlsanity.com/paper/1804.07954/) (2018)
- [Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays](https://mlsanity.com/paper/2102.00781/) (2021)
- [Prompt- and Trait Relation-aware Cross-prompt Essay Trait Scoring](https://mlsanity.com/paper/2305.16826/) (2023)
- [Distilling ChatGPT for Explainable Automated Student Answer Assessment](https://mlsanity.com/paper/2305.12962/) (2023)
- [Autoregressive Score Generation for Multi-trait Essay Scoring](https://mlsanity.com/paper/2403.08332/) (2024)

## Built on by (most cited first, 22 of 22)

- [AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition](https://mlsanity.com/paper/2509.21910/) (2025)
- [Teach-to-Reason with Scoring: Self-Explainable Rationale-Driven Multi-Trait Essay Scoring](https://mlsanity.com/paper/2502.20748/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback](https://mlsanity.com/paper/2601.04574/) (2026)
- [EssayBench: Evaluating Large Language Models in Multi-Genre Chinese Essay Writing](https://mlsanity.com/paper/2506.02596/) (2025)
- [Agreement Between Large Language Models and Human Raters in Essay Scoring: A Research Synthesis](https://mlsanity.com/paper/2512.14561/) (2025)
- [Exploring the Utilities of the Rationales from Large Language Models to Enhance Automated Essay Scoring](https://mlsanity.com/paper/2510.27131/) (2025)
- [Enhancing Automated Essay Scoring With Three Techniques: Two-Stage Fine-Tuning, Score Alignment, and Self-Training](https://mlsanity.com/paper/2602.01747/) (2026)
- [Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring](https://mlsanity.com/paper/2605.25731/) (2026)
- [SWIM: Student Writing Simulation via Proficiency-Conditioned Generation](https://mlsanity.com/paper/2609.03215/) (2026)
- [Composable Cross-prompt Essay Scoring by Merging Models](https://mlsanity.com/paper/2505.18548/) (2025)
- [Zero-Shot Grammar Competency Estimation Using Large Language Model Generated Pseudo Labels](https://mlsanity.com/paper/2511.13152/) (2025)
- [FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing](https://mlsanity.com/paper/2511.22883/) (2025)
- [Specialists or Generalists? Multi-Agent and Single-Agent LLMs for Essay Grading](https://mlsanity.com/paper/2601.22386/) (2026)
- [Automated Multiple Mini Interview (MMI) Scoring](https://mlsanity.com/paper/2602.02360/) (2026)
- [Decision-Level Ordinal Modeling for Multimodal Essay Scoring with Large Language Models](https://mlsanity.com/paper/2603.14891/) (2026)
- [REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading](https://mlsanity.com/paper/2605.27402/) (2026)
- [Incentives Of EdTech: A Systematic Review Of EduNLP Research](https://mlsanity.com/paper/2606.13691/) (2026)
- [When Rubrics Change: Cross-Rubric Generalization for Critical Thinking Essay Scoring](https://mlsanity.com/paper/2607.13433/) (2026)
- [WrAFT: a Modularized Automated Writing Evaluation System for Argumentative Essays](https://mlsanity.com/paper/2607.14524/) (2026)
- [Learning to Grade Efficiently: A Bandit-Driven Prompt-Selection Framework for Low-Cost LLM Essay Scoring](https://mlsanity.com/paper/2608.23814/) (2026)
- [A Unified Framework to Elicit Structured Feedback for Interpretable Multi-Trait Essay Scoring](https://mlsanity.com/paper/2608.28407/) (2026)

## Abstract (arXiv)

Existing automated essay scoring (AES) has solely relied on essay text without using explanatory rationales for the scores, thereby forgoing an opportunity to capture the specific aspects evaluated by rubric indicators in a fine-grained manner. This paper introduces Rationale-based Multiple Trait Scoring (RMTS), a novel approach for multi-trait essay scoring that integrates prompt-engineering-based large language models (LLMs) with a fine-tuning-based essay scoring model using a smaller large language model (S-LLM). RMTS uses an LLM-based trait-wise rationale generation system where a separate LLM agent generates trait-specific rationales based on rubric guidelines, which the scoring model uses to accurately predict multi-trait scores. Extensive experiments on benchmark datasets, including ASAP, ASAP++, and Feedback Prize, show that RMTS significantly outperforms state-of-the-art models and vanilla S-LLMs in trait-specific scoring. By assisting quantitative assessment with fine-grained qualitative rationales, RMTS enhances the trait-wise reliability, providing partial explanations about essays. The code is available at https://github.com/BBeeChu/RMTS.git.