# Enhancing Critical Thinking in Education by means of a Socratic Chatbot arXiv: https://arxiv.org/abs/2409.05511 ยท HTML: https://mlsanity.com/paper/2409.05511/ ## Key facts - Cited by 21 papers in the map (data through 14 Oct 2026). - Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. - Builds on 7 of its 42 references that are in the map. - First posted on arXiv 9 Sep 2024. ## Placed in - LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/) ## Builds on (most cited first, 7 of 7) - [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [Automatic Generation of Socratic Subquestions for Teaching Math Word Problems](https://mlsanity.com/paper/2211.12835/) (2022) - [Recent Advances in Neural Question Generation](https://mlsanity.com/paper/1905.08949/) (2019) - [Synthetic Dialogue Dataset Generation using LLM Agents](https://mlsanity.com/paper/2401.17461/) (2024) - [Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation](https://mlsanity.com/paper/2305.14386/) (2023) - [Response Generation for Cognitive Behavioral Therapy with Large Language Models: Comparative Study with Socratic Questioning](https://mlsanity.com/paper/2401.15966/) (2024) ## Built on by (most cited first, 21 of 21) - [Position: LLMs Can be Good Tutors in English Education](https://mlsanity.com/paper/2502.05467/) (2025) - [Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMs](https://mlsanity.com/paper/2508.06583/) (2025) - [ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference](https://mlsanity.com/paper/2509.14537/) (2025) - [How Problematic Writer-AI Interactions (Rather than Problematic AI) Hinder Writers' Idea Generation](https://mlsanity.com/paper/2503.11915/) (2025) - [Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models](https://mlsanity.com/paper/2505.11341/) (2025) - [Iffy-Or-Not: Extending the Web to Support the Critical Evaluation of Fallacious Texts](https://mlsanity.com/paper/2503.14412/) (2025) - [Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach](https://mlsanity.com/paper/2505.09576/) (2025) - [ConvoLearn: A Learning Sciences Grounded Dataset for Fine-Tuning Dialogic AI Tutors](https://mlsanity.com/paper/2601.08950/) (2026) - [Hey Chat, Can You Teach Me? Structuring Socratic Dialogue for Human Learning in the Wild](https://mlsanity.com/paper/2606.11744/) (2026) - [AITEE -- Agentic Tutor for Electrical Engineering](https://mlsanity.com/paper/2505.21582/) (2025) - [ELLIS Alicante at CQs-Gen 2025: Winning the critical thinking questions shared task: LLM-based question generation and selection](https://mlsanity.com/paper/2506.14371/) (2025) - [Human-AI collaboration or obedient and often clueless AI in instruct, serve, repeat dynamics?](https://mlsanity.com/paper/2508.10919/) (2025) - [LLM Chatbot-Creation Approaches](https://mlsanity.com/paper/2509.13326/) (2025) - [Examining Student Interactions with a Pedagogical AI-Assistant for Essay Writing and their Impact on Students Writing Quality](https://mlsanity.com/paper/2512.08596/) (2025) - [MetaCues: Enabling Critical Engagement with Generative AI for Information Seeking and Sensemaking](https://mlsanity.com/paper/2603.19634/) (2026) - [BLADE: Better Language Answers through Dialogue and Explanations](https://mlsanity.com/paper/2604.03236/) (2026) - [SCAN: A Decision-Making Framework for Effective Task Allocation with Generative AI](https://mlsanity.com/paper/2606.15601/) (2026) - [Curiosity as Linguistic Intervention: Using LLM Tutoring Dialogues to Influence Exploratory Learning Behavior](https://mlsanity.com/paper/2606.22349/) (2026) - [Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming](https://mlsanity.com/paper/2607.03303/) (2026) - [The Theoretical Foundation of Socratic Tests: Dynamic, Multimodal, Conversational Examinations](https://mlsanity.com/paper/2607.29624/) (2026) - [Investigating the Impacts of Generative AI on Information Seeking](https://mlsanity.com/paper/2609.14638/) (2026) ## Abstract (arXiv) While large language models (LLMs) are increasingly playing a pivotal role in education by providing instantaneous, adaptive responses, their potential to promote critical thinking remains understudied. In this paper, we fill such a gap and present an innovative educational chatbot designed to foster critical thinking through Socratic questioning. Unlike traditional intelligent tutoring systems, including educational chatbots, that tend to offer direct answers, the proposed Socratic tutor encourages students to explore various perspectives and engage in self-reflection by posing structured, thought-provoking questions. Our Socratic questioning is implemented by fine and prompt-tuning the open-source pretrained LLM with a specialized dataset that stimulates critical thinking and offers multiple viewpoints. In an effort to democratize access and to protect the students' privacy, the proposed tutor is based on small LLMs (Llama2 7B and 13B-parameter models) that are able to run locally on off-the-shelf hardware. We validate our approach in a battery of experiments consisting of interactions between a simulated student and the chatbot to evaluate its effectiveness in enhancing critical thinking skills. Results indicate that the Socratic tutor supports the development of reflection and critical thinking significantly better than standard chatbots. Our approach opens the door for improving educational outcomes by cultivating active learning and encouraging intellectual autonomy.