# How Students Use AI Feedback Matters: Experimental Evidence on Physics Achievement and Autonomy

arXiv: https://arxiv.org/abs/2505.08672 · HTML: https://mlsanity.com/paper/2505.08672/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2025.

## Built on by (most cited first, 2 of 2)

- [A Meta-Analysis of LLM Effects on Students across Qualification, Socialisation, and Subjectification](https://mlsanity.com/paper/2509.22725/) (2025)
- [Student Development Agent: Risk-free Simulation for Evaluating AIED Innovations](https://mlsanity.com/paper/2510.09183/) (2025)

## Abstract (arXiv)

Despite the precision and adaptiveness of generative AI (GAI)-powered feedback provided to students, existing practice and literature might ignore how usage patterns impact student learning. This study examines the heterogeneous effects of GAI-powered personalized feedback on high school students' physics achievement and autonomy through two randomized controlled trials, with a major focus on usage patterns. Each experiment lasted for five weeks, involving a total of 387 students. Experiment 1 (n = 121) assessed compulsory usage of the personalized recommendation system, revealing that low-achieving students significantly improved academic performance (d = 0.673, p < 0.05) when receiving AI-generated heuristic solution hints, whereas medium-achieving students' performance declined (d = -0.539, p < 0.05) with conventional answers provided by workbook. Notably, high-achieving students experienced a significant decline in self-regulated learning (d = -0.477, p < 0.05) without any significant gains in achievement. Experiment 2 (n = 266) investigated the usage pattern of autonomous on-demand help, demonstrating that fully learner-controlled AI feedback significantly enhanced academic performance for high-achieving students (d = 0.378, p < 0.05) without negatively impacting their autonomy. However, autonomy notably declined among lower achievers exposed to on-demand AI interventions (d = -0.383, p < 0.05), particularly in the technical-psychological dimension (d = -0.549, p < 0.05), which has a large overlap with self-regulation. These findings underscore the importance of usage patterns when applying GAI-powered personalized feedback to students.