# Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use Scale

arXiv: https://arxiv.org/abs/2512.12413 · HTML: https://mlsanity.com/paper/2512.12413/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 172 references that are in the map.
- First posted on arXiv 13 Dec 2025.

## Builds on (most cited first, 17 of 17)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding](https://mlsanity.com/paper/2304.10548/) (2023)
- [LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding](https://mlsanity.com/paper/2306.14924/) (2023)
- [Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance](https://mlsanity.com/paper/2412.09315/) (2024)
- [Students' Voices on Generative AI: Perceptions, Benefits, and Challenges in Higher Education](https://mlsanity.com/paper/2305.00290/) (2023)
- [Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction](https://mlsanity.com/paper/2303.09224/) (2023)
- [The Perils & Promises of Fact-checking with Large Language Models](https://mlsanity.com/paper/2310.13549/) (2023)
- [Global AI Ethics: A Review of the Social Impacts and Ethical Implications of Artificial Intelligence](https://mlsanity.com/paper/1907.07892/) (2019)
- [The Role of Deductive and Inductive Reasoning in Large Language Models](https://mlsanity.com/paper/2410.02892/) (2024)
- [ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience](https://mlsanity.com/paper/2307.01135/) (2023)
- [Protecting Human Cognition in the Age of AI](https://mlsanity.com/paper/2502.12447/) (2025)
- [Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction](https://mlsanity.com/paper/2409.16708/) (2024)
- [GLAT: The Generative AI Literacy Assessment Test](https://mlsanity.com/paper/2411.00283/) (2024)
- [Generative AI Literacy: A Comprehensive Framework for Literacy and Responsible Use](https://mlsanity.com/paper/2504.19038/) (2025)

## Built on by (most cited first, 1 of 1)

- [Beyond AI Literacy: A Structured Review and Exploratory Meta-Analysis of Measures for Competent Generative-AI Use](https://mlsanity.com/paper/2609.15624/) (2026)

## Abstract (arXiv)

Generative AI tools are increasingly embedded in everyday work and learning, yet their fluency, opacity, and propensity to hallucinate mean that users must critically evaluate AI outputs rather than accept them at face value. The present research conceptualises critical thinking in AI use as a dispositional tendency to verify the source and content of AI-generated information, to understand how models work and where they fail, and to reflect on the broader implications of relying on AI. Across six studies (N = 1341), we developed and validated the 13-item critical thinking in AI use scale and mapped its nomological network. Study 1 generated and content-validated scale items. Study 2 supported a three-factor structure (Verification, Motivation, and Reflection). Studies 3 and 4 confirmed the higher-order model, demonstrated strong factor loadings, internal consistency, sex invariance, convergent and discriminant evidence for validity, and showed that critical thinking in AI use was positively associated with openness, extraversion, positive trait affect, and frequency of AI use. Study 5 supported the scale's test-retest reliability. Lastly, Study 6 demonstrated criterion evidence of validity for the scale, with higher critical thinking in AI use scores predicting more frequent and diverse verification strategies, greater veracity judgement accuracy in a novel and naturalistic GPT-powered AI chatbot fact-checking task, and deeper reflection about responsible AI. The current work clarifies why and how people exercise oversight over generative AI outputs and provides a validated scale and ecologically grounded paradigm to support theory testing, cross-group, and longitudinal research on critical thinking in AI use.