mlsanityData to 7 Oct 2026

A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science

arXiv 2403.14565 ↗First posted 21 Mar 2024cs.CLcited by 31

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

This paper explores the use of large language models (LLMs) to score and explain short-answer assessments in K-12 science. While existing methods can score more structured math and computer science assessments, they often do not provide explanations for the scores. Our study focuses on employing GPT-4 for automated assessment in middle school Earth Science, combining few-shot and active learning with chain-of-thought reasoning. Using a human-in-the-loop approach, we successfully score and provide meaningful explanations for formative assessment responses. A systematic analysis of our method's pros and cons sheds light on the potential for human-in-the-loop techniques to enhance automated grading for open-ended science assessments.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 39 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

11 of 39 references.

References · 11

  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  3. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  4. 2020A Survey of Deep Active Learningcited by 547
  5. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
  6. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  7. 2023Assessing the Quality of Multiple-Choice Questions Using GPT-4 and Rule-Based Methodscited by 23
  8. 2023Does Informativeness Matter? Active Learning for Educational Dialogue Act Classificationcited by 8
  9. 2022Giving Feedback on Interactive Student Programs with Meta-Explorationcited by 6
  10. 2021Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guaranteescited by 5
  11. 2023H-AES: Towards Automated Essay Scoring for Hindicited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 19; 2026: 4.

31 papers.

Sort:

By area:

  1. 2025EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Frameworkcited by 23
  2. 2024A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimizationcited by 16
  3. 2025Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizationscited by 14
  4. 2025Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotationscited by 14
  5. 2024Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoringcited by 13
  6. 2024Towards A Human-in-the-Loop LLM Approach to Collaborative Discourse Analysiscited by 7
  7. 2024Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Modelscited by 6
  8. 2025Personalizing Student-Agent Interactions Using Log-Contextualized Retrieval-Augmented Generation (RAG)cited by 6
  9. 2024"My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essayscited by 5
  10. 2025A Theory of Adaptive Scaffolding for LLM-Based Pedagogical Agentscited by 5
  11. 2024Multimodal Methods for Analyzing Learning and Training Environments: A Systematic Literature Reviewcited by 4
  12. 2025CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedbackcited by 4
19 more
  1. 2025CoDAE: Adapting Large Language Models for Education via Chain-of-Thought Data Augmentationcited by 4
  2. 2025Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identificationcited by 3
  3. 2025LLMs as Educational Analysts: Transforming Multimodal Data Traces into Actionable Reading Assessment Reportscited by 3
  4. 2024Incorporating Metabolic Information into LLMs for Anomaly Detection in Clinical Time-Seriescited by 2
  5. 2025XiHeFusion: Harnessing Large Language Models for Science Communication in Nuclear Fusioncited by 2
  6. 2026Evidence-Decision-Feedback: Theory-Driven Adaptive Scaffolding for LLM Agentscited by 2
  7. 2025Operationalizing Automated Essay Scoring: A Human-Aware Approachcited by 1
  8. 2025InqEduAgent: Adaptive AI Learning Partners with Gaussian Process Augmentationcited by 1
  9. 2025LearnLens: An AI-Enhanced Dashboard to Support Teachers in Open-Ended Classroomscited by 1
  10. 2025RubiSCoT: A Framework for AI-Supported Academic Assessmentcited by 1
  11. 2026How Uncertain Is the Grade? A Benchmark of Uncertainty Metrics for LLM-Based Automatic Assessmentcited by 1
  12. 2026AI Misuse in Education Is a Measurement Problem: Toward a Learning Visibility Frameworkcited by 1
  13. 2026Implicit Grading Bias in Large Language Models: How Writing Style Affects Automated Assessment Across Math, Programming, and Essay Taskscited by 1
  14. 2024Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracycited by 0
  15. 2025PEMUTA: Pedagogically-Enriched Multi-Granular Undergraduate Thesis Assessmentcited by 0
  16. 2025Designing Gaze Analytics for ELA Instruction: A User-Centered Dashboard with Conversational AI Supportcited by 0
  17. 2025Bridging Cultural Distance Between Models Default and Local Classroom Demands: How Global Teachers Adopt GenAI to Support Everyday Teaching Practicescited by 0
  18. 2025Populism Meets AI: Advancing Populism Research with LLMscited by 0
  19. 2025AI-Enabled grading with near-domain data for scaling feedback with human-level accuracycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Applying Large Language Models and Chain-of-Thought for Automatic Scoring
  2. 2026A Benchmark for LLM's Understanding of Middle School and High School Science Topics
  3. 2025Enhancing LLM-Based Short Answer Grading with Retrieval-Augmented Generation
  4. 2025A Report on the llms evaluating the high school questions
  5. 2025Evaluating GPT- and Reasoning-based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Implications for Educational Assessment
  6. 2023Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
  7. 2023Investigating the Efficacy of Large Language Models in Reflective Assessment Methods through Chain of Thoughts Prompting
  8. 2025LearnLens: LLM-Enabled Personalised, Curriculum-Grounded Feedback with Educators in the Loop