mlsanityData to 7 Oct 2026

Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation

arXiv 2404.15845 ↗First posted 24 Apr 2024cs.CLcited by 63

Not on the map. Why?

Abstract (arXiv)

Individual feedback can help students improve their essay writing skills. However, the manual effort required to provide such feedback limits individualization in practice. Automatically-generated essay feedback may serve as an alternative to guide students at their own pace, convenience, and desired frequency. Large language models (LLMs) have demonstrated strong performance in generating coherent and contextually relevant text. Yet, their ability to provide helpful essay feedback is unclear. This work explores several prompting strategies for LLM-based zero-shot and few-shot generation of essay feedback. Inspired by Chain-of-Thought prompting, we study how and to what extent automated essay scoring (AES) can benefit the quality of generated feedback. We evaluate both the AES performance that LLMs can achieve with prompting only and the helpfulness of the generated essay feedback. Our results suggest that tackling AES and feedback generation jointly improves AES performance. However, while our manual evaluation emphasizes the quality of the generated essay feedback, the impact of essay scoring on the generated feedback remains low ultimately.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 13 of its 59 references that are in the map. First posted on arXiv 24 Apr 2024.

Builds on

13 of 59 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  5. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  6. 2016Automatic Text Scoring Using Neural Networkscited by 51
  7. 2023The language of prompting: What linguistic properties make a prompt successful?cited by 43
  8. 2018Automated essay scoring with string kernels and word embeddingscited by 31
  9. 2018Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Inputcited by 16
  10. 2016Automated assessment of non-native learner essays: Investigating the role of linguistic featurescited by 12
  11. 2022Sentence-level Feedback Generation for English Language Learners: Does Data Augmentation Help?cited by 3
  12. 2022Error syntax aware augmentation of feedback comment generation datasetcited by 3
1 more
  1. 2023Review of feedback in Automated Essay Scoringcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 28; 2026: 25.

63 papers.

Sort:

By area:

  1. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  2. 2024Can AI grade your essays? A comparative analysis of large language models and teacher ratings in multidimensional essay scoringcited by 19
  3. 2025Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgmentscited by 16
  4. 2025Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creationcited by 13
  5. 2025TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoringcited by 12
  6. 2024Automated Text Scoring in the Age of Generative AI for the GPU-poorcited by 10
  7. 2024Evaluating GPT-4 at Grading Handwritten Solutions in Math Examscited by 10
  8. 2025LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writingcited by 10
  9. 2025Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judgecited by 10
  10. 2025Improve LLM-based Automatic Essay Scoring with Linguistic Featurescited by 8
  11. 2024Grading Massive Open Online Courses Using Large Language Modelscited by 7
  12. 2025Position: LLMs Can be Good Tutors in English Educationcited by 6
38 more
  1. 2025Teach-to-Reason with Scoring: Self-Explainable Rationale-Driven Multi-Trait Essay Scoringcited by 6
  2. 2025Opportunities and Challenges of LLMs in Education: An NLP Perspectivecited by 6
  3. 2024"My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essayscited by 5
  4. 2025Does the Prompt-based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring?cited by 5
  5. 2025ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanationscited by 4
  6. 2026FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedbackcited by 4
  7. 2025SEFL: A Framework for Generating Synthetic Educational Assignment Feedback with LLM Agentscited by 3
  8. 2026Meta-Judging with Large Language Models: Concepts, Methods, and Challengescited by 3
  9. 2024Grammatical Error Feedback: An Implicit Evaluation Approachcited by 2
  10. 2025Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detectioncited by 2
  11. 2025Identifying Legal Holdings with LLMs: A Systematic Study of Performance, Scale, and Memorizationcited by 2
  12. 2025Evaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classroomscited by 2
  13. 2025MAGIC: Multi-Agent Argumentation and Grammar Integrated Critiquercited by 2
  14. 2026LLMs Do Not Grade Essays Like Humanscited by 2
  15. 2026PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedbackcited by 2
  16. 2024Development of REGAI: Rubric Enabled Generative Artificial Intelligencecited by 1
  17. 2024EvalYaks: Instruction Tuning Datasets and LoRA Fine-tuned Models for Automated Scoring of CEFR B2 Speaking Assessment Transcriptscited by 1
  18. 2024On the Suitability of pre-trained foundational LLMs for Analysis in German Legal Educationcited by 1
  19. 2025On the effectiveness of LLMs for automatic grading of open-ended questions in Spanishcited by 1
  20. 2025SAS-Bench: A Fine-Grained Benchmark for Evaluating Short Answer Scoring with Large Language Modelscited by 1
  21. 2025FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoringcited by 1
  22. 2026Enhancing Automated Essay Scoring With Three Techniques: Two-Stage Fine-Tuning, Score Alignment, and Self-Trainingcited by 1
  23. 2026The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errorscited by 1
  24. 2026LLM-Based Scientific Peer Review: Methods, Benchmarks, and Reliability Challengescited by 1
  25. 2026SEFORA: Student Essays with Feedback Corpus and LLM Feedback Evaluation Frameworkcited by 1
  26. 2025Validity Arguments For Constructed Response Scoring Using Generative Artificial Intelligence Applicationscited by 0
  27. 2025Euskarazko lehen C1 ebaluatzaile automatikoacited by 0
  28. 2025Leveraging AI Graders for Missing Score Imputation to Achieve Accurate Ability Estimation in Constructed-Response Testscited by 0
  29. 2025Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Modelscited by 0
  30. 2025MTQ-Eval: Multilingual Text Quality Evaluation for Language Modelscited by 0
  31. 2025Zero-Shot Grammar Competency Estimation Using Large Language Model Generated Pseudo Labelscited by 0
  32. 2025FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writingcited by 0
  33. 2025Automatic Essay Scoring and Feedback Generation in Basque Language Learningcited by 0
  34. 2025Agreement Between Large Language Models and Human Raters in Essay Scoring: A Research Synthesiscited by 0
  35. 2026Hybrid Neural-LLM Pipeline for Morphological Glossing in Endangered Language Documentation: A Case Study of Jungar Tuvancited by 0
  36. 2026From Prompting to Preference Optimization: A Comparative Study of LLM-based Automated Essay Scoringcited by 0
  37. 2026Decision-Level Ordinal Modeling for Multimodal Essay Scoring with Large Language Modelscited by 0
  38. 2026From Feature-Based Models to Generative AI: Validity Evidence for Constructed Response Scoringcited by 0

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards
  2. 2026FOXGLOVE: Understanding Goal-Oriented and Anchored Writing Feedback from Experts and LLMs on Argumentative Essays
  3. 2025Exploring LLM-Generated Feedback for Economics Essays: How Teaching Assistants Evaluate and Envision Its Use
  4. 2024Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions
  5. 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models
  6. 2025"Can You See Me Think?" Grounding LLM Feedback in Keystrokes and Revision Patterns
  7. 2026Student Evaluation of Repeated AI Feedback Across a Semester of Writing ↗
  8. 2026Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback