# Evaluating Feedback Focus and Pedagogical Adaptivity in LLM-Generated Feedback on Student Writing

arXiv: https://arxiv.org/abs/2609.28026 · HTML: https://mlsanity.com/paper/2609.28026/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 64 references that are in the map.
- First posted on arXiv 23 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues](https://mlsanity.com/paper/2205.07540/) (2022)
- [The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs](https://mlsanity.com/paper/2501.10970/) (2025)
- [Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation](https://mlsanity.com/paper/2404.15845/) (2024)
- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge](https://mlsanity.com/paper/2410.03775/) (2024)
- [A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice](https://mlsanity.com/paper/2404.16958/) (2024)
- [Large Language Models Do Multi-Label Classification Differently](https://mlsanity.com/paper/2505.17510/) (2025)
- [Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedback](https://mlsanity.com/paper/2507.16007/) (2025)
- [LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writing](https://mlsanity.com/paper/2502.11368/) (2025)
- [TeachLM: Post-Training LLMs for Education Using Authentic Learning Data](https://mlsanity.com/paper/2510.05087/) (2025)
- [Towards Adaptive Feedback with AI: Comparing the Feedback Quality of LLMs and Teachers on Experimentation Protocols](https://mlsanity.com/paper/2502.12842/) (2025)
- [Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback](https://mlsanity.com/paper/2511.08225/) (2025)
- [LLM Review: Enhancing Creative Writing via Blind Peer Review Feedback](https://mlsanity.com/paper/2601.08003/) (2026)
- [FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback](https://mlsanity.com/paper/2601.04574/) (2026)
- [Grammatical Error Feedback: An Implicit Evaluation Approach](https://mlsanity.com/paper/2408.09565/) (2024)
- [Beyond Agreement: Rethinking Ground Truth in Educational AI Annotation](https://mlsanity.com/paper/2508.00143/) (2025)
- [Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback](https://mlsanity.com/paper/2603.12471/) (2026)
- [Evaluation of ChatGPT Feedback on ELL Writers' Coherence and Cohesion](https://mlsanity.com/paper/2310.06505/) (2023)
- [SEFORA: Student Essays with Feedback Corpus and LLM Feedback Evaluation Framework](https://mlsanity.com/paper/2607.00274/) (2026)

## Abstract (arXiv)

We investigate whether state-of-the-art large language models (LLMs) generate feedback that reflects the pedagogical practices of expert teachers in terms of feedback focus and adaptivity. Previous evaluation efforts have examined feedback characteristics, its impact on learning, and its target, yet the focus of feedback and its adaptivity remains largely overlooked. To bridge this gap, we adopt and refine Narciss's taxonomy into seven feedback focus types to annotate teacher and LLM-generated feedback across three university writing courses. We release FeedType, a benchmark containing annotated teacher and LLM feedback from six LLMs under three prompting strategies. We assess the coverage and distribution of feedback focus types, and examine whether LLMs adapt their feedback across draft stages and student performance levels as an expert instructor does. Our findings show that while most LLMs cover most feedback focus types, they fail to reflect teacher feedback distributions and show varying levels of adaptivity, with none matching the teachers' adaptive behavior. We believe FeedType will support future research on pedagogical alignment in LLM feedback generation.