mlsanityData to 7 Oct 2026

Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction

arXiv 2306.03090 ↗First posted 5 Jun 2023cs.CL cs.AIcited by 37

Not on the map. Why?

Abstract (arXiv)

Coaching, which involves classroom observation and expert feedback, is a widespread and fundamental part of teacher training. However, the majority of teachers do not have access to consistent, high quality coaching due to limited resources and access to expertise. We explore whether generative AI could become a cost-effective complement to expert feedback by serving as an automated teacher coach. In doing so, we propose three teacher coaching tasks for generative AI: (A) scoring transcript segments based on classroom observation instruments, (B) identifying highlights and missed opportunities for good instructional strategies, and (C) providing actionable suggestions for eliciting more student reasoning. We recruit expert math teachers to evaluate the zero-shot performance of ChatGPT on each of these tasks for elementary math classroom transcripts. Our results reveal that ChatGPT generates responses that are relevant to improving instruction, but they are often not novel or insightful. For example, 82% of the model's suggestions point to places in the transcript where the teacher is already implementing that suggestion. Our work highlights the challenges of producing insightful, novel and truthful feedback for teachers while paving the way for future research to address these obstacles and improve the capacity of generative AI to coach teachers.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 13 of its 42 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

13 of 42 references.

References · 13

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  6. 2022Teaching language models to support answers with verified quotescited by 290
  7. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  8. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  9. 2023ChatGPT: Beginning of an End of Manual Linguistic Data Annotation? Use Case of Automatic Genre Identificationcited by 45
  10. 2022The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcriptscited by 43
  11. 2023Learning gain differences between ChatGPT and human tutor generated algebra hintscited by 42
  12. 2021Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Applicationcited by 10
1 more
  1. 2023Better by you, better than me, chatgpt3 as writing assistance in students essayscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 17; 2025: 3; 2026: 5.

37 papers.

Sort:

By area:

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  3. 2023Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakescited by 51
  4. 2024Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertisecited by 46
  5. 2023Grounding Gaps in Language Model Generationscited by 45
  6. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Educationcited by 38
  7. 2023LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interactioncited by 28
  8. 2024Formal Theorem Proving by Rewarding LLMs to Decompose Proofs Hierarchicallycited by 22
  9. 2023A Material Lens on Coloniality in NLPcited by 19
  10. 2023Flesch or Fumble? Evaluating Readability Standard Alignment of Instruction-Tuned Language Modelscited by 17
  11. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  12. 2023Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challengescited by 13
25 more
  1. 2024Edu-ConvoKit: An Open-Source Library for Education Conversation Datacited by 11
  2. 2024Improving Assessment of Tutoring Practices using Retrieval-Augmented Generationcited by 11
  3. 2024Evaluating and Optimizing Educational Content with Large Language Model Judgmentscited by 11
  4. 2024Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errorscited by 9
  5. 2024The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Educationcited by 9
  6. 2023Automated Evaluation of Classroom Instructional Support with LLMs and BoWs: Connecting Global Predictions to Specific Feedbackcited by 7
  7. 2024A Survey on Human-AI Collaboration with Large Foundation Modelscited by 7
  8. 2024Automated Assessment of Encouragement and Warmth in Classrooms Leveraging Multimodal Emotional Features and ChatGPTcited by 7
  9. 2024LLMs in Education: Novel Perspectives, Challenges, and Opportunitiescited by 7
  10. 2025The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Coursecited by 7
  11. 2025Opportunities and Challenges of LLMs in Education: An NLP Perspectivecited by 6
  12. 2024When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challengescited by 4
  13. 2024Analyzing Large Language Models for Classroom Discussion Assessmentcited by 4
  14. 2023"Mistakes Help Us Grow": Facilitating and Evaluating Growth Mindset Supportive Language in Classroomscited by 2
  15. 2025Multimodal Assessment of Classroom Discourse Quality: A Text-Centered Attention-Based Multi-Task Learning Approachcited by 2
  16. 2026Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impactcited by 2
  17. 2023Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translationcited by 1
  18. 2023Exploring the Capabilities of ChatGPT in Ancient Chinese Translation and Person Name Recognitioncited by 0
  19. 2024Enhancing Instructional Quality: Leveraging Computer-Assisted Textual Analysis to Generate In-Depth Insights from Educational Artifactscited by 0
  20. 2024Constrained C-Test Generation via Mixed-Integer Programmingcited by 0
  21. 2024Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturerscited by 0
  22. 2026Mitigating LLM biases toward spurious social contexts using direct preference optimizationcited by 0
  23. 2026Teachers' Perceived Benefits and Risks of AI Across Fifty-Five Countries: An Audit of LLM Alignment and Steerabilitycited by 0
  24. 2026From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessmentcited by 0
  25. 2026EduPanel: A Three-Agent LLM Judge for Teaching Videos -- Reliability, Complementarity, and Human Trust Calibrationcited by 0