# How Can I Improve? Using GPT to Highlight the Desired and Undesired Parts of Open-ended Responses arXiv: https://arxiv.org/abs/2405.00291 ยท HTML: https://mlsanity.com/paper/2405.00291/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 15 of its 55 references that are in the map. - First posted on arXiv 1 May 2024. ## Builds on (most cited first, 15 of 15) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021) - [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022) - [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021) - [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018) - [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023) - [Fine-tuning ChatGPT for Automatic Scoring](https://mlsanity.com/paper/2310.10072/) (2023) - [Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning](https://mlsanity.com/paper/2308.03234/) (2023) - [Improving Assessment of Tutoring Practices using Retrieval-Augmented Generation](https://mlsanity.com/paper/2402.14594/) (2024) - [AUC Maximization for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2212.04800/) (2022) - [Using Large Language Models to Provide Explanatory Feedback to Human Tutors](https://mlsanity.com/paper/2306.15498/) (2023) - [Re-weighting Tokens: A Simple and Effective Active Learning Strategy for Named Entity Recognition](https://mlsanity.com/paper/2311.00906/) (2023) - [A Boundary Regression Model for Nested Named Entity Recognition](https://mlsanity.com/paper/2011.14330/) (2020) - [Low-Resource Named Entity Recognition: Can One-vs-All AUC Maximization Help?](https://mlsanity.com/paper/2311.04918/) (2023) ## Built on by (most cited first, 10 of 10) - [Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study](https://mlsanity.com/paper/2504.05570/) (2025) - [How Can I Get It Right? Using GPT to Rephrase Incorrect Trainee Responses](https://mlsanity.com/paper/2405.00970/) (2024) - [Does Multiple Choice Have a Future in the Age of Generative AI? A Posttest-only RCT](https://mlsanity.com/paper/2412.10267/) (2024) - [Enhancing the De-identification of Personally Identifiable Information in Educational Data](https://mlsanity.com/paper/2501.09765/) (2025) - [Do Tutors Learn from Equity Training and Can Generative AI Assess It?](https://mlsanity.com/paper/2412.11255/) (2024) - [LLM-Generated Feedback Supports Learning If Learners Choose to Use It](https://mlsanity.com/paper/2506.17006/) (2025) - [LLM Prompt Evaluation for Educational Applications](https://mlsanity.com/paper/2601.16134/) (2026) - [Examining GPT's Capability to Generate and Map Course Concepts and Their Relationship](https://mlsanity.com/paper/2504.08856/) (2025) - [Chat-Based Support Alone May Not Be Enough: Comparing Conversational and Embedded LLM Feedback for Mathematical Proof Learning](https://mlsanity.com/paper/2602.18807/) (2026) - [AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice](https://mlsanity.com/paper/2606.18617/) (2026) ## Abstract (arXiv) Automated explanatory feedback systems play a crucial role in facilitating learning for a large cohort of learners by offering feedback that incorporates explanations, significantly enhancing the learning process. However, delivering such explanatory feedback in real-time poses challenges, particularly when high classification accuracy for domain-specific, nuanced responses is essential. Our study leverages the capabilities of large language models, specifically Generative Pre-Trained Transformers (GPT), to explore a sequence labeling approach focused on identifying components of desired and less desired praise for providing explanatory feedback within a tutor training dataset. Our aim is to equip tutors with actionable, explanatory feedback during online training lessons. To investigate the potential of GPT models for providing the explanatory feedback, we employed two commonly-used approaches: prompting and fine-tuning. To quantify the quality of highlighted praise components identified by GPT models, we introduced a Modified Intersection over Union (M-IoU) score. Our findings demonstrate that: (1) the M-IoU score effectively correlates with human judgment in evaluating sequence quality; (2) using two-shot prompting on GPT-3.5 resulted in decent performance in recognizing effort-based (M-IoU of 0.46) and outcome-based praise (M-IoU of 0.68); and (3) our optimally fine-tuned GPT-3.5 model achieved M-IoU scores of 0.64 for effort-based praise and 0.84 for outcome-based praise, aligning with the satisfaction levels evaluated by human coders. Our results show promise for using GPT models to provide feedback that focuses on specific elements in their open-ended responses that are desirable or could use improvement.