# Prompting Large Language Models With the Socratic Method

arXiv: https://arxiv.org/abs/2303.08769 · HTML: https://mlsanity.com/paper/2303.08769/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 63 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [Coarse-to-Fine Decoding for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04793/) (2018)
- [BERTese: Learning to Speak to BERT](https://mlsanity.com/paper/2103.05327/) (2021)
- [Word-Level Coreference Resolution](https://mlsanity.com/paper/2109.04127/) (2021)
- [Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing](https://mlsanity.com/paper/2110.15534/) (2021)
- [Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning](https://mlsanity.com/paper/2210.12587/) (2022)
- [A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representation](https://mlsanity.com/paper/2009.07968/) (2020)

## Built on by (most cited first, 30 of 43)

- [BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games](https://mlsanity.com/paper/2411.13543/) (2024)
- [OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling](https://mlsanity.com/paper/2407.09887/) (2024)
- [SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning](https://mlsanity.com/paper/2503.11951/) (2025)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought](https://mlsanity.com/paper/2308.08614/) (2023)
- [SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation](https://mlsanity.com/paper/2310.00074/) (2023)
- [Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity](https://mlsanity.com/paper/2402.06529/) (2024)
- [Mathematical Language Models: A Survey](https://mlsanity.com/paper/2312.07622/) (2023)
- [Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM](https://mlsanity.com/paper/2403.08010/) (2024)
- [Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling](https://mlsanity.com/paper/2411.00750/) (2024)
- [Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoning](https://mlsanity.com/paper/2503.01642/) (2025)
- [An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide](https://mlsanity.com/paper/2402.14837/) (2024)
- [What Makes a Good Natural Language Prompt?](https://mlsanity.com/paper/2506.06950/) (2025)
- [RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates](https://mlsanity.com/paper/2506.11083/) (2025)
- [EVINCE: Optimizing Multi-LLM Dialogues Using Conditional Statistics and Information Theory](https://mlsanity.com/paper/2408.14575/) (2024)
- [ALAS: A Stateful Multi-LLM Agent Framework for Disruption-Aware Planning](https://mlsanity.com/paper/2505.12501/) (2025)
- [Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMs](https://mlsanity.com/paper/2508.06583/) (2025)
- [Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Tasks](https://mlsanity.com/paper/2410.24032/) (2024)
- [Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimation](https://mlsanity.com/paper/2510.08044/) (2025)
- [Helpful or Harmful? Exploring the Efficacy of Large Language Models for Online Grooming Prevention](https://mlsanity.com/paper/2403.09795/) (2024)
- [Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences](https://mlsanity.com/paper/2507.08440/) (2025)
- [Science in the Era of ChatGPT, Large Language Models and Generative AI: Challenges for Research Ethics and How to Respond](https://mlsanity.com/paper/2305.15299/) (2023)
- [Team Trifecta at Factify5WQA: Setting the Standard in Fact Verification with Fine-Tuning](https://mlsanity.com/paper/2403.10281/) (2024)
- [Uncovering Biases with Reflective Large Language Models](https://mlsanity.com/paper/2408.13464/) (2024)
- [Dialogic Pedagogy for Large Language Models: Aligning Conversational AI with Proven Theories of Learning](https://mlsanity.com/paper/2506.19484/) (2025)
- [Multi-Agent Collaborative Intelligence: Dual-Dial Control for Reliable LLM Reasoning](https://mlsanity.com/paper/2510.04488/) (2025)
- [Evolutionary Reinforcement Learning based AI tutor for Socratic Interdisciplinary Instruction](https://mlsanity.com/paper/2512.11930/) (2025)
- [Modeling Emotions and Ethics with Large Language Models](https://mlsanity.com/paper/2404.13071/) (2024)
- [Integrating Emotional and Linguistic Models for Ethical Compliance in Large Language Models](https://mlsanity.com/paper/2405.07076/) (2024)
- [A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression](https://mlsanity.com/paper/2407.00065/) (2024)

## Abstract (arXiv)

This paper presents a systematic approach to using the Socratic method in developing prompt templates that effectively interact with large language models, including GPT-3. Various methods are examined, and those that yield precise answers and justifications while fostering creativity and imagination to enhance creative writing are identified. Techniques such as em definition, em elenchus, em dialectic, em maieutics, em generalization, and em counterfactual reasoning are discussed for their application in engineering prompt templates and their connections to inductive, deductive, and abductive reasoning. Through examples, the effectiveness of these dialogue and reasoning methods is demonstrated. An interesting observation is made that when the task's goal and user intent are conveyed to GPT-3 via ChatGPT before the start of a dialogue, the large language model seems to connect to the external context expressed in the intent and perform more effectively.