# To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks

arXiv: https://arxiv.org/abs/2606.30549 · HTML: https://mlsanity.com/paper/2606.30549/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 18 of 18)

- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [The Robots are Here: Navigating the Generative AI Revolution in Computing Education](https://mlsanity.com/paper/2310.00658/) (2023)
- ["It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers](https://mlsanity.com/paper/2304.02491/) (2023)
- [How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment](https://mlsanity.com/paper/2309.14049/) (2023)
- [The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers](https://mlsanity.com/paper/2405.17739/) (2024)
- [The Effects of Generative AI on Computing Students' Help-Seeking Preferences](https://mlsanity.com/paper/2401.02262/) (2024)
- [Generative AI in Computing Education: Perspectives of Students and Instructors](https://mlsanity.com/paper/2308.04309/) (2023)
- [CS1-LLM: Integrating LLMs into CS1 Instruction](https://mlsanity.com/paper/2406.15379/) (2024)
- [Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools](https://mlsanity.com/paper/2412.14732/) (2024)
- [The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Programming Tasks](https://mlsanity.com/paper/2506.10051/) (2025)
- [Early Performance Prediction using Interpretable Patterns in Programming Process Data](https://mlsanity.com/paper/2102.05765/) (2021)
- [Students' Perceptions and Use of Generative AI Tools for Programming Across Different Computing Courses](https://mlsanity.com/paper/2410.06865/) (2024)
- [Synthetic Students: A Comparative Study of Bug Distribution Between Large Language Models and Computing Students](https://mlsanity.com/paper/2410.09193/) (2024)
- [Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023](https://mlsanity.com/paper/2411.11166/) (2024)
- [The Evolving Usage of GenAI by Computing Students](https://mlsanity.com/paper/2412.16453/) (2024)
- [Beyond the Benefits: A Systematic Review of the Harms and Consequences of Generative AI in Computing Education](https://mlsanity.com/paper/2510.04443/) (2025)
- [KOALA: a Configurable Tool for Collecting IDE Data When Solving Programming Tasks](https://mlsanity.com/paper/2506.21266/) (2025)
- [On the Opportunities of Large Language Models for Programming Process Data](https://mlsanity.com/paper/2411.00414/) (2024)

## Abstract (arXiv)

AI code completion tools, such as Github Copilot, provide students with code suggestions to help them write programs. However, recent qualitative studies suggest that students fail to critically evaluate these suggestions. We present Clover, a code completion tool that logs students' interactions with code suggestions and additionally offers attention checks to probe reflective engagement during programming tasks. We also develop a taxonomy of behavioral interaction metrics for AI-assisted programming, informed by literature. We analyzed relationships between interaction patterns, engagement with attention checks, and task performance. We observed that higher rates of tab accept were associated with lower attention check performance, while increased dwell time was associated with higher attention check performance. We conclude by discussing how programming process data and attention checks might support reflective engagement in AI-assisted programming.