# Understanding Help-Seeking Behavior of Students Using LLMs vs. Web Search for Writing SQL Queries

arXiv: https://arxiv.org/abs/2408.08401 · HTML: https://mlsanity.com/paper/2408.08401/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Builds on (most cited first, 9 of 9)

- [The Metacognitive Demands and Opportunities of Generative AI](https://mlsanity.com/paper/2312.10893/) (2023)
- [The Robots are Here: Navigating the Generative AI Revolution in Computing Education](https://mlsanity.com/paper/2310.00658/) (2023)
- [CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs](https://mlsanity.com/paper/2401.11314/) (2024)
- [Computing Education in the Era of Generative AI](https://mlsanity.com/paper/2306.02608/) (2023)
- [Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment](https://mlsanity.com/paper/2307.03744/) (2023)
- [GPTutor: a ChatGPT-powered programming tool for code explanation](https://mlsanity.com/paper/2305.01863/) (2023)
- [Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms](https://mlsanity.com/paper/2406.07571/) (2024)
- [ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience](https://mlsanity.com/paper/2307.01135/) (2023)
- [Comparing Traditional and LLM-based Search for Image Geolocation](https://mlsanity.com/paper/2401.10184/) (2024)

## Built on by (most cited first, 2 of 2)

- [Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models](https://mlsanity.com/paper/2504.05325/) (2025)
- [Rethinking Search: A Study of University Students' Perspectives on Using LLMs and Traditional Search Engines in Academic Problem Solving](https://mlsanity.com/paper/2510.17726/) (2025)

## Abstract (arXiv)

Growth in the use of large language models (LLMs) in programming education is altering how students write SQL queries. Traditionally, students relied heavily on web search for coding assistance, but this has shifted with the adoption of LLMs like ChatGPT. However, the comparative process and outcomes of using web search versus LLMs for coding help remain underexplored. To address this, we conducted a randomized interview study in a database classroom to compare web search and LLMs, including a publicly available LLM (ChatGPT) and an instructor-tuned LLM, for writing SQL queries. Our findings indicate that using an instructor-tuned LLM required significantly more interactions than both ChatGPT and web search, but resulted in a similar number of edits to the final SQL query. No significant differences were found in the quality of the final SQL queries between conditions, although the LLM conditions directionally showed higher query quality. Furthermore, students using instructor-tuned LLM reported a lower mental demand. These results have implications for learning and productivity in programming education.