# Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoring

arXiv: https://arxiv.org/abs/2504.05736 · HTML: https://mlsanity.com/paper/2504.05736/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 13 of 13)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://mlsanity.com/paper/2304.00723/) (2023)
- [Process Reward Model with Q-Value Rankings](https://mlsanity.com/paper/2410.11287/) (2024)
- [From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples](https://mlsanity.com/paper/2404.07544/) (2024)
- [A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models](https://mlsanity.com/paper/2310.09497/) (2023)
- [Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs](https://mlsanity.com/paper/2401.06431/) (2024)
- [Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring](https://mlsanity.com/paper/2404.04941/) (2024)
- [Prompt Agnostic Essay Scorer: A Domain Generalization Approach to Cross-prompt Automated Essay Scoring](https://mlsanity.com/paper/2008.01441/) (2020)
- [Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input](https://mlsanity.com/paper/1804.06898/) (2018)
- [A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss](https://mlsanity.com/paper/2210.10305/) (2022)

## Built on by (most cited first, 6 of 6)

- [An overview of artificial intelligence in computer-assisted language learning](https://mlsanity.com/paper/2505.02032/) (2025)
- [Agreement Between Large Language Models and Human Raters in Essay Scoring: A Research Synthesis](https://mlsanity.com/paper/2512.14561/) (2025)
- [Using Large Language Models to Assess Teachers' Pedagogical Content Knowledge](https://mlsanity.com/paper/2505.19266/) (2025)
- [ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities](https://mlsanity.com/paper/2508.14377/) (2025)
- [From Feature-Based Models to Generative AI: Validity Evidence for Constructed Response Scoring](https://mlsanity.com/paper/2603.19280/) (2026)
- [Cost-efficient generative AI summarization for scalable automated essay scoring in educational assessment](https://mlsanity.com/paper/2607.15829/) (2026)

## Abstract (arXiv)

In recent years, large language models (LLMs) achieve remarkable success across a variety of tasks. However, their potential in the domain of Automated Essay Scoring (AES) remains largely underexplored. Moreover, compared to English data, the methods for Chinese AES is not well developed. In this paper, we propose Rank-Then-Score (RTS), a fine-tuning framework based on large language models to enhance their essay scoring capabilities. Specifically, we fine-tune the ranking model (Ranker) with feature-enriched data, and then feed the output of the ranking model, in the form of a candidate score set, with the essay content into the scoring model (Scorer) to produce the final score. Experimental results on two benchmark datasets, HSK and ASAP, demonstrate that RTS consistently outperforms the direct prompting (Vanilla) method in terms of average QWK across all LLMs and datasets, and achieves the best performance on Chinese essay scoring using the HSK dataset.