The Impact of AI Usage and Informativeness on Skill Development in Logical Reasoning
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Artificial intelligence (AI) is being increasingly integrated into human problem-solving, yet its effects on individual skill development remain unclear. We examine how both AI usage and informativeness can shape learning in the context of a controlled logical reasoning task with on-demand access to AI assistance. We find that greater AI usage is associated with weaker skill development: heavy AI users underperform relative to comparable peers, whereas light AI users perform similarly to matched users who do not use AI. We also find in our study that these patterns are mediated by AI informativeness. Low-information AI neither improves immediate performance nor preserves performance after AI assistance is removed, and is linked to weaker learning overall. On the other hand, high-information AI was found to improve short-run performance without reducing post-AI outcomes on average in our experiments, but with heterogeneous effects. Our findings in general suggest that AI can, depending on context, either complement human skill development by amplifying independent reasoning or can act as a substitute that undermines such reasoning, with the implication that regulating AI access and usage will be important for promoting skill development in the presence of AI assistance.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 44 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
14 of 44 references.
References · 14
- 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
- 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
- 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
- 2020Learning to Complement Humanscited by 123
- 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
- 2025Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Taskcited by 103
- 2022Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learningcited by 53
- 2022Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imagingcited by 32
- 2026How AI Impacts Skill Formationcited by 20
- 2023Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressurecited by 17
- 2024Designing for Appropriate Reliance: The Roles of AI Uncertainty Presentation, Initial User Decision, and User Demographics in AI-Assisted Decision-Makingcited by 11
- 2024Explainable AI improves task performance in human-AI collaborationcited by 11
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026AI Assistance Reduces Persistence and Hurts Independent Performance
- 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail
- 2026Agentivism: a learning theory for the age of artificial intelligence
- 2026Supporting Calibrated Reliance in Human-AI Collaboration: Different Strategies for Different Tasks
- 2026Skill Use or Skill Theater? Evaluating the Reasoning Backroom in Skill-Augmented Language Agents
- 2023Studying and improving reasoning in humans and machines
- 2026To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks
- 2026AI Assistants Overassist