# Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks

arXiv: https://arxiv.org/abs/2605.04534 · HTML: https://mlsanity.com/paper/2605.04534/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 4 of its 21 references that are in the map.
- First posted on arXiv 6 May 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 4 of 4)

- [The Prompt Report: A Systematic Survey of Prompt Engineering Techniques](https://mlsanity.com/paper/2406.06608/) (2024)
- [TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks](https://mlsanity.com/paper/2305.11430/) (2023)
- [Assigning AI: Seven Approaches for Students, with Prompts](https://mlsanity.com/paper/2306.10052/) (2023)
- [An Empirical Study to Understand How Students Use ChatGPT for Writing Essays](https://mlsanity.com/paper/2501.10551/) (2025)

## Built on by (most cited first, 1 of 1)

- [DraftTrace: A Multi-View Analytics Environment for AI-Integrated Writing](https://mlsanity.com/paper/2609.36544/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are becoming increasingly embedded in students' learning practices, yet much of what is known about how students use LLMs and how this usage impacts learning comes from problem-solving domains or constrained experimental settings. We present an analysis of data on LLM usage collected during two offerings of a research-oriented course where students learn to read, reason about, and critique academic papers. Without restrictions on whether or how to use LLMs, students reported their LLM usage practices when asked to do these activities as a series of homework assignments during the course. This paper extends prior work done on data from a single offering of the same course by presenting a refined bottom-up categorization of LLM usage types, cross-labeled by the extent of student initiative these usages entail. Furthermore, we examine how LLM use impacts student learning, measured by performance on three midterms, looking at factors such as frequency and type of usage.