# Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation

arXiv: https://arxiv.org/abs/2404.15845 · HTML: https://mlsanity.com/paper/2404.15845/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 59 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Automatic Text Scoring Using Neural Networks](https://mlsanity.com/paper/1606.04289/) (2016)
- [The language of prompting: What linguistic properties make a prompt successful?](https://mlsanity.com/paper/2311.01967/) (2023)
- [Automated essay scoring with string kernels and word embeddings](https://mlsanity.com/paper/1804.07954/) (2018)
- [Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input](https://mlsanity.com/paper/1804.06898/) (2018)
- [Automated assessment of non-native learner essays: Investigating the role of linguistic features](https://mlsanity.com/paper/1612.00729/) (2016)
- [Sentence-level Feedback Generation for English Language Learners: Does Data Augmentation Help?](https://mlsanity.com/paper/2212.08999/) (2022)
- [Error syntax aware augmentation of feedback comment generation dataset](https://mlsanity.com/paper/2212.14293/) (2022)
- [Review of feedback in Automated Essay Scoring](https://mlsanity.com/paper/2307.05553/) (2023)

## Built on by (most cited first, 30 of 63)

- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [Can AI grade your essays? A comparative analysis of large language models and teacher ratings in multidimensional essay scoring](https://mlsanity.com/paper/2411.16337/) (2024)
- [Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgments](https://mlsanity.com/paper/2504.17087/) (2025)
- [Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creation](https://mlsanity.com/paper/2502.18357/) (2025)
- [TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring](https://mlsanity.com/paper/2505.14577/) (2025)
- [Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge](https://mlsanity.com/paper/2505.19477/) (2025)
- [Automated Text Scoring in the Age of Generative AI for the GPU-poor](https://mlsanity.com/paper/2407.01873/) (2024)
- [Evaluating GPT-4 at Grading Handwritten Solutions in Math Exams](https://mlsanity.com/paper/2411.05231/) (2024)
- [LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writing](https://mlsanity.com/paper/2502.11368/) (2025)
- [Improve LLM-based Automatic Essay Scoring with Linguistic Features](https://mlsanity.com/paper/2502.09497/) (2025)
- [Grading Massive Open Online Courses Using Large Language Models](https://mlsanity.com/paper/2406.11102/) (2024)
- [Position: LLMs Can be Good Tutors in English Education](https://mlsanity.com/paper/2502.05467/) (2025)
- [Teach-to-Reason with Scoring: Self-Explainable Rationale-Driven Multi-Trait Essay Scoring](https://mlsanity.com/paper/2502.20748/) (2025)
- [Does the Prompt-based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring?](https://mlsanity.com/paper/2504.21330/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- ["My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays](https://mlsanity.com/paper/2409.07453/) (2024)
- [ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations](https://mlsanity.com/paper/2506.14200/) (2025)
- [FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback](https://mlsanity.com/paper/2601.04574/) (2026)
- [SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agents](https://mlsanity.com/paper/2502.12927/) (2025)
- [Meta-Judging with Large Language Models: Concepts, Methods, and Challenges](https://mlsanity.com/paper/2601.17312/) (2026)
- [Grammatical Error Feedback: An Implicit Evaluation Approach](https://mlsanity.com/paper/2408.09565/) (2024)
- [On the Suitability of pre-trained foundational LLMs for Analysis in German Legal Education](https://mlsanity.com/paper/2412.15902/) (2024)
- [Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection](https://mlsanity.com/paper/2502.06487/) (2025)
- [Identifying Legal Holdings with LLMs: A Systematic Study of Performance, Scale, and Memorization](https://mlsanity.com/paper/2505.02172/) (2025)
- [Evaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classrooms](https://mlsanity.com/paper/2506.04822/) (2025)
- [MAGIC: Multi-Agent Argumentation and Grammar Integrated Critiquer](https://mlsanity.com/paper/2506.13037/) (2025)
- [Agreement Between Large Language Models and Human Raters in Essay Scoring: A Research Synthesis](https://mlsanity.com/paper/2512.14561/) (2025)
- [LLMs Do Not Grade Essays Like Humans](https://mlsanity.com/paper/2603.23714/) (2026)
- [PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback](https://mlsanity.com/paper/2606.20287/) (2026)
- [Development of REGAI: Rubric Enabled Generative Artificial Intelligence](https://mlsanity.com/paper/2408.02811/) (2024)

## Abstract (arXiv)

Individual feedback can help students improve their essay writing skills. However, the manual effort required to provide such feedback limits individualization in practice. Automatically-generated essay feedback may serve as an alternative to guide students at their own pace, convenience, and desired frequency. Large language models (LLMs) have demonstrated strong performance in generating coherent and contextually relevant text. Yet, their ability to provide helpful essay feedback is unclear. This work explores several prompting strategies for LLM-based zero-shot and few-shot generation of essay feedback. Inspired by Chain-of-Thought prompting, we study how and to what extent automated essay scoring (AES) can benefit the quality of generated feedback. We evaluate both the AES performance that LLMs can achieve with prompting only and the helpfulness of the generated essay feedback. Our results suggest that tackling AES and feedback generation jointly improves AES performance. However, while our manual evaluation emphasizes the quality of the generated essay feedback, the impact of essay scoring on the generated feedback remains low ultimately.