# Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms

arXiv: https://arxiv.org/abs/2406.07571 · HTML: https://mlsanity.com/paper/2406.07571/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2024.

## Built on by (most cited first, 20 of 20)

- [PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation](https://mlsanity.com/paper/2409.12538/) (2024)
- [Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation](https://mlsanity.com/paper/2502.02780/) (2025)
- [Planning of Heuristics: Strategic Planning on Large Language Models with Monte Carlo Tree Search for Automating Heuristic Optimization](https://mlsanity.com/paper/2502.11422/) (2025)
- [Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recovery](https://mlsanity.com/paper/2412.11656/) (2024)
- [ExploreSelf: Fostering User-driven Exploration and Reflection on Personal Challenges with Adaptive Guidance by Large Language Models](https://mlsanity.com/paper/2409.09662/) (2024)
- [Large Language Model Agents for Improving Engagement with Behavior Change Interventions: Application to Digital Mindfulness](https://mlsanity.com/paper/2407.13067/) (2024)
- [Understanding Before Reasoning: Enhancing Chain-of-Thought with Iterative Summarization Pre-Prompting](https://mlsanity.com/paper/2501.04341/) (2025)
- [ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference](https://mlsanity.com/paper/2509.14537/) (2025)
- [Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations](https://mlsanity.com/paper/2512.22883/) (2025)
- [Understanding Help-Seeking Behavior of Students Using LLMs vs. Web Search for Writing SQL Queries](https://mlsanity.com/paper/2408.08401/) (2024)
- [MedSimAI: Simulation and Formative Feedback Generation to Enhance Deliberate Practice in Medical Education](https://mlsanity.com/paper/2503.05793/) (2025)
- [When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring](https://mlsanity.com/paper/2603.27076/) (2026)
- [MathBuddy: A Multimodal System for Affective Math Tutoring](https://mlsanity.com/paper/2508.19993/) (2025)
- ["I made this (sort of)": Negotiating authorship, confronting fraudulence, and exploring new musical spaces with prompt-based AI music generation](https://mlsanity.com/paper/2507.23365/) (2025)
- [Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning](https://mlsanity.com/paper/2510.05417/) (2025)
- [Designing and Evaluating Chain-of-Hints for Scientific Question Answering](https://mlsanity.com/paper/2510.21087/) (2025)
- [CogMCTS: A Novel Cognitive-Guided Monte Carlo Tree Search Framework for Iterative Heuristic Evolution with Large Language Models](https://mlsanity.com/paper/2512.08609/) (2025)
- [Hybrid LLM-Embedded Dialogue Agents for Learner Reflection: Designing Responsive and Theory-Driven Interactions](https://mlsanity.com/paper/2602.20486/) (2026)
- [Moving Beyond Review: Applying Language Models to Planning and Translation in Reflection](https://mlsanity.com/paper/2603.28596/) (2026)
- [It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluation](https://mlsanity.com/paper/2607.17627/) (2026)

## Abstract (arXiv)

Self-reflection on learning experiences constitutes a fundamental cognitive process, essential for the consolidation of knowledge and the enhancement of learning efficacy. However, traditional methods to facilitate reflection often face challenges in personalization, immediacy of feedback, engagement, and scalability. Integration of Large Language Models (LLMs) into the reflection process could mitigate these limitations. In this paper, we conducted two randomized field experiments in undergraduate computer science courses to investigate the potential of LLMs to help students engage in post-lesson reflection. In the first experiment (N=145), students completed a take-home assignment with the support of an LLM assistant; half of these students were then provided access to an LLM designed to facilitate self-reflection. The results indicated that the students assigned to LLM-guided reflection reported increased self-confidence and performed better on a subsequent exam two weeks later than their peers in the control condition. In the second experiment (N=112), we evaluated the impact of LLM-guided self-reflection against other scalable reflection methods, such as questionnaire-based activities and review of key lecture slides, after assignment. Our findings suggest that the students in the questionnaire and LLM-based reflection groups performed equally well and better than those who were only exposed to lecture slides, according to their scores on a proctored exam two weeks later on the same subject matter. These results underscore the utility of LLM-guided reflection and questionnaire-based activities in improving learning outcomes. Our work highlights that focusing solely on the accuracy of LLMs can overlook their potential to enhance metacognitive skills through practices such as self-reflection. We discuss the implications of our research for the Edtech community.