# Promoting Critical Thinking With Domain-Specific Generative AI Provocations

arXiv: https://arxiv.org/abs/2603.19975 · HTML: https://mlsanity.com/paper/2603.19975/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Builds on (most cited first, 10 of 10)

- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks](https://mlsanity.com/paper/2310.07879/) (2023)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Understanding User Perceptions, Collaborative Experience and User Engagement in Different Human-AI Interaction Designs for Co-Creative Systems](https://mlsanity.com/paper/2204.13217/) (2022)
- [Towards evaluating and eliciting high-quality documentation for intelligent systems](https://mlsanity.com/paper/2011.08774/) (2020)
- [Language Models as Critical Thinking Tools: A Case Study of Philosophers](https://mlsanity.com/paper/2404.04516/) (2024)
- [Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks](https://mlsanity.com/paper/2504.17421/) (2025)

## Abstract (arXiv)

The evidence on the effects of generative AI (GenAI) on critical thinking is mixed, with studies suggesting both potential harms and benefits depending on its implementation. Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking. Drawing on our experience designing and evaluating two GenAI-powered tools for knowledge work, ArtBot in the domain of fine art interpretation and Privy in the domain of AI privacy, we reflect on how design decisions shape the form and effectiveness of such provocations. Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking. We present participant experiences with both prototypes and discuss how supporting critical thinking may require moving beyond static provocations toward approaches that adapt to user preferences and levels of expertise.