# A Large Language Model Approach to Educational Survey Feedback Analysis arXiv: https://arxiv.org/abs/2309.17447 ยท HTML: https://mlsanity.com/paper/2309.17447/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 26 of its 102 references that are in the map. - First posted on arXiv 29 Sep 2023. ## Builds on (most cited first, 26 of 26) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022) - [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022) - [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022) - [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023) - [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019) - [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023) - [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020) - [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023) - [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023) - [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023) - [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021) - [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023) - [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023) - [Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech](https://mlsanity.com/paper/2302.07736/) (2023) - [Efficient Few-Shot Learning Without Prompts](https://mlsanity.com/paper/2209.11055/) (2022) - [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022) - [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023) - [ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning](https://mlsanity.com/paper/2304.06588/) (2023) - [Automated Annotation with Generative AI Requires Validation](https://mlsanity.com/paper/2306.00176/) (2023) - [Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remark](https://mlsanity.com/paper/2304.11085/) (2023) - [A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis](https://mlsanity.com/paper/2301.08826/) (2023) - [Sentiment analysis and opinion mining on educational data: A survey](https://mlsanity.com/paper/2302.04359/) (2023) - [In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workers](https://mlsanity.com/paper/2112.02255/) (2021) ## Built on by (most cited first, 14 of 14) - [AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot](https://mlsanity.com/paper/2604.13940/) (2026) - [Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions](https://mlsanity.com/paper/2602.14878/) (2026) - [Large Language Models in Computer Science Education: A Systematic Literature Review](https://mlsanity.com/paper/2410.16349/) (2024) - [Beyond Text-to-Text: An Overview of Multimodal and Generative Artificial Intelligence for Education Using Topic Modeling](https://mlsanity.com/paper/2409.16376/) (2024) - [Personalizing Student-Agent Interactions Using Log-Contextualized Retrieval-Augmented Generation (RAG)](https://mlsanity.com/paper/2505.17238/) (2025) - [From Moderation to Mediation: Can LLMs Serve as Mediators in Online Flame Wars?](https://mlsanity.com/paper/2512.03005/) (2025) - [MM-CRITIC: A Holistic Evaluation of Large Multimodal Models as Multimodal Critique](https://mlsanity.com/paper/2511.09067/) (2025) - [A Data Synthesis Method Driven by Large Language Models for Proactive Mining of Implicit User Intentions in Tourism](https://mlsanity.com/paper/2505.11533/) (2025) - [DMN-Guided Prompting: A Framework for Controlling LLM Behavior](https://mlsanity.com/paper/2505.11701/) (2025) - [Leveraging Interview-Informed LLMs to Model Survey Responses: Comparative Insights from AI-Generated and Human Data](https://mlsanity.com/paper/2505.21997/) (2025) - [Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program](https://mlsanity.com/paper/2508.05513/) (2025) - [RescueLens: LLM-Powered Triage and Action on Volunteer Feedback for Food Rescue](https://mlsanity.com/paper/2511.15698/) (2025) - [A Benchmarking Methodology to Assess Open-Source Video Large Language Models in Automatic Captioning of News Videos](https://mlsanity.com/paper/2603.27662/) (2026) - [BPMN4CAI: A BPMN Extension for Modeling Dynamic Conversational AI](https://mlsanity.com/paper/2608.27149/) (2026) ## Abstract (arXiv) This paper assesses the potential for the large language models (LLMs) GPT-4 and GPT-3.5 to aid in deriving insight from education feedback surveys. Exploration of LLM use cases in education has focused on teaching and learning, with less exploration of capabilities in education feedback analysis. Survey analysis in education involves goals such as finding gaps in curricula or evaluating teachers, often requiring time-consuming manual processing of textual responses. LLMs have the potential to provide a flexible means of achieving these goals without specialized machine learning models or fine-tuning. We demonstrate a versatile approach to such goals by treating them as sequences of natural language processing (NLP) tasks including classification (multi-label, multi-class, and binary), extraction, thematic analysis, and sentiment analysis, each performed by LLM. We apply these workflows to a real-world dataset of 2500 end-of-course survey comments from biomedical science courses, and evaluate a zero-shot approach (i.e., requiring no examples or labeled training data) across all tasks, reflecting education settings, where labeled data is often scarce. By applying effective prompting practices, we achieve human-level performance on multiple tasks with GPT-4, enabling workflows necessary to achieve typical goals. We also show the potential of inspecting LLMs' chain-of-thought (CoT) reasoning for providing insight that may foster confidence in practice. Moreover, this study features development of a versatile set of classification categories, suitable for various course types (online, hybrid, or in-person) and amenable to customization. Our results suggest that LLMs can be used to derive a range of insights from survey text.