# AI Assistants Overassist

arXiv: https://arxiv.org/abs/2607.21306 · HTML: https://mlsanity.com/paper/2607.21306/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 23 of 23)

- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://mlsanity.com/paper/2302.06590/) (2023)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving](https://mlsanity.com/paper/2405.12205/) (2024)
- [The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues](https://mlsanity.com/paper/2205.07540/) (2022)
- [Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions](https://mlsanity.com/paper/2312.02913/) (2023)
- [Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluation](https://mlsanity.com/paper/2405.11591/) (2024)
- [Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant](https://mlsanity.com/paper/2502.01390/) (2025)
- [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022)
- [How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?](https://mlsanity.com/paper/2404.03302/) (2024)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students](https://mlsanity.com/paper/2410.04078/) (2024)
- [A large language model-assisted education tool to provide feedback on open-ended responses](https://mlsanity.com/paper/2308.02439/) (2023)
- [Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors](https://mlsanity.com/paper/2510.11502/) (2025)
- [Who's in Charge? Disempowerment Patterns in Real-World LLM Usage](https://mlsanity.com/paper/2601.19062/) (2026)
- [COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis](https://mlsanity.com/paper/2408.05006/) (2024)
- [Modeling Student Learning with 3.8 Million Program Traces](https://mlsanity.com/paper/2510.05056/) (2025)
- [AI Assistance Reduces Persistence and Hurts Independent Performance](https://mlsanity.com/paper/2604.04721/) (2026)
- [More AI Assistance Reduces Cognitive Engagement: Examining the AI Assistance Dilemma in AI-Supported Note-Taking](https://mlsanity.com/paper/2509.03392/) (2025)
- [Extending the Hint Factory for the assistance dilemma: A novel, data-driven HelpNeed Predictor for proactive problem-solving help](https://mlsanity.com/paper/2010.04124/) (2020)
- [When not to help: planning for lasting human-AI collaboration](https://mlsanity.com/paper/2508.01837/) (2025)
- [Learning When to Quit in Sales Conversations](https://mlsanity.com/paper/2511.01181/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used as tutors and thought partners, helping users reason through problems. While guidance from AI assistants can scaffold thinking and foster learning, such benefits depend on how they help--for instance, intervening too early or too frequently may hinder true learning and cognitive engagement. Yet how AI systems navigate intervention decisions during problem-solving remains poorly understood. Here, we introduce Int-Bench, a simulation-based benchmark for evaluating LLM interventions during learning. Int-Bench simulates a "student" solving a problem while a "teacher" monitors the student's reasoning and decides whether, when, and how to intervene. Across three domains--code debugging, mathematics, and brain teasers--we evaluate LLM teachers on the frequency and timing of interventions, as well as their impact on both immediate task success and generalization to new problems. We also compare LLMs to humans, finding that LLMs intervene more frequently and earlier than humans. Moreover, in contrast to humans, they tend to provide complete solutions rather than targeted hints. These findings suggest that current LLM assistants often optimize for short-term success rather than supporting the reasoning processes needed for deeper learning and long-term success.