Cost-efficient generative AI summarization for scalable automated essay scoring in educational assessment
Not on the map. Why?
Abstract (arXiv)
Automated essay scoring (AES) enables scalable assessment and timely feedback but remains challenged by transformer input-length limitations, which can cause information loss when processing long essays. This study proposes a generative AI-assisted summarization framework to improve long-form essay representation while maintaining scoring reliability. Using the ASAP 2.0 dataset, we generate controlled-length summaries with three GPT-5 variants (GPT-5, GPT-5 mini, and GPT-5 nano) and use them as inputs for downstream AES models. To preserve original writing signals, handcrafted linguistic features extracted from full essays are integrated with summary representations to form a hybrid framework. The approach is evaluated in terms of scoring performance, summarization quality, and computational cost. Scoring reliability is measured using quadratic weighted kappa (QWK), while summary quality is assessed through lexical overlap, semantic similarity, information retention, and redundancy metrics. Results show that GPT-5 mini achieves the highest agreement with human ratings, whereas GPT-5 produces the strongest summarization quality. Summary quality decreases for higher-scoring essays, indicating that more complex writing is more difficult to compress without information loss. These findings reveal trade-offs among model capacity, summary fidelity, cost efficiency, and preservation of educational constructs. This study provides an initial controlled evaluation of GPT-based summarization for AES and identifies important baselines and ablation studies required for future generalization. Overall, generative AI summarization offers a promising approach for scalable writing assessment while requiring careful validation of information preservation and fairness.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 54 references that are in the map. First posted on arXiv 17 Jul 2026.
Builds on
22 of 54 references.
References · 22
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2023GPT-4 Technical Reportcited by 18,396
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
- 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
10 more
- 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 244
- 2023Applying Large Language Models and Chain-of-Thought for Automatic Scoringcited by 69
- 2016Automatic Text Scoring Using Neural Networkscited by 51
- 2023Fine-tuning ChatGPT for Automatic Scoringcited by 51
- 2024Can Large Language Models Automatically Score Proficiency of Written Essays?cited by 35
- 2025LCES: Zero-shot Automated Essay Scoring via Pairwise Comparisons Using Large Language Modelscited by 8
- 2025Rank-Then-Score: Enhancing Large Language Models for Automated Essay Scoringcited by 6
- 2025Exploration of Summarization by Generative Language Models for Automated Scoring of Long Essayscited by 2
- 2026LLM Essay Scoring Under Holistic and Analytic Rubrics: Prompt Effects and Biascited by 2
- 2026PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedbackcited by 2
Built on by
Not cited yet.