mlsanityData to 7 Oct 2026

Prompting Large Language Models With the Socratic Method

arXiv 2303.08769 ↗First posted 17 Feb 2023cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

This paper presents a systematic approach to using the Socratic method in developing prompt templates that effectively interact with large language models, including GPT-3. Various methods are examined, and those that yield precise answers and justifications while fostering creativity and imagination to enhance creative writing are identified. Techniques such as em definition, em elenchus, em dialectic, em maieutics, em generalization, and em counterfactual reasoning are discussed for their application in engineering prompt templates and their connections to inductive, deductive, and abductive reasoning. Through examples, the effectiveness of these dialogue and reasoning methods is demonstrated. An interesting observation is made that when the task's goal and user intent are conveyed to GPT-3 via ChatGPT before the start of a dialogue, the large language model seems to connect to the external context expressed in the intent and perform more effectively.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 24 of its 63 references that are in the map. First posted on arXiv 17 Feb 2023.

Builds on

24 of 63 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2019How Can We Know What Language Models Know?cited by 953
  9. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  10. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  11. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  12. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
12 more
  1. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  2. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  3. 2023Augmented Language Models: a Surveycited by 398
  4. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  5. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  6. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  7. 2018Coarse-to-Fine Decoding for Neural Semantic Parsingcited by 110
  8. 2021BERTese: Learning to Speak to BERTcited by 82
  9. 2021Word-Level Coreference Resolutioncited by 40
  10. 2021Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsingcited by 25
  11. 2022Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuningcited by 9
  12. 2020A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 16; 2025: 18; 2026: 4.

43 papers.

Sort:

By area:

  1. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  2. 2024OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modelingcited by 66
  3. 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planningcited by 58
  4. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thoughtcited by 41
  5. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  6. 2023SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluationcited by 27
  7. 2024Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguitycited by 26
  8. 2023Mathematical Language Models: A Surveycited by 24
  9. 2024Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLMcited by 21
  10. 2024Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Samplingcited by 13
  11. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8
  12. 2024An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guidecited by 7
31 more
  1. 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debatescited by 7
  2. 2024EVINCE: Optimizing Multi-LLM Dialogues Using Conditional Statistics and Information Theorycited by 6
  3. 2025What Makes a Good Natural Language Prompt?cited by 6
  4. 2025ALAS: A Stateful Multi-LLM Agent Framework for Disruption-Aware Planningcited by 5
  5. 2025Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMscited by 5
  6. 2024Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Taskscited by 4
  7. 2025Towards Reliable LLM-based Robot Planning via Combined Uncertainty Estimationcited by 4
  8. 2025Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferencescited by 3
  9. 2023Science in the Era of ChatGPT, Large Language Models and Generative AI: Challenges for Research Ethics and How to Respondcited by 2
  10. 2024Team Trifecta at Factify5WQA: Setting the Standard in Fact Verification with Fine-Tuningcited by 2
  11. 2025Dialogic Pedagogy for Large Language Models: Aligning Conversational AI with Proven Theories of Learningcited by 2
  12. 2025Multi-Agent Collaborative Intelligence: Dual-Dial Control for Reliable LLM Reasoningcited by 2
  13. 2025Evolutionary Reinforcement Learning based AI tutor for Socratic Interdisciplinary Instructioncited by 2
  14. 2024Helpful or Harmful? Exploring the Efficacy of Large Language Models for Online Grooming Preventioncited by 1
  15. 2024Modeling Emotions and Ethics with Large Language Modelscited by 1
  16. 2024Integrating Emotional and Linguistic Models for Ethical Compliance in Large Language Modelscited by 1
  17. 2024A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regressioncited by 1
  18. 2024Uncovering Biases with Reflective Large Language Modelscited by 1
  19. 2024VG-TVP: Multimodal Procedural Planning via Visually Grounded Text-Video Promptingcited by 1
  20. 2025Self-reflecting Large Language Models: A Hegelian Dialectical Approachcited by 1
  21. 2025Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approachcited by 1
  22. 2025AGI Requires a Coordination Layer on Top of Pattern Repositoriescited by 1
  23. 2023Brave new world: Artificial Intelligence in teaching and learningcited by 0
  24. 2024Ensuring Ground Truth Accuracy in Healthcare with the EVINCE frameworkcited by 0
  25. 2025Investigating the Effectiveness of a Socratic Chain-of-Thoughts Reasoning Method for Task Planning in Robotics, A Case Studycited by 0
  26. 2025P-CoT: A Pedagogically-motivated Participatory Chain-of-Thought Prompting for Phonological Reasoning in LLMscited by 0
  27. 2025Closing the Expression Gap in LLM Instructions via Socratic Questioningcited by 0
  28. 2026The Art of Socratic Inquiry: A Framework for Proactive Template-Guided Therapeutic Conversation Generationcited by 0
  29. 2026TRACE: An Operational Reasoning Schema for Auditable Agentic Commitmentscited by 0
  30. 2026Investigating the Impacts of Generative AI on Information Seekingcited by 0
  31. 2026Multi-Agent Debate for Explainable Trading: Reasoning, Consensus, and Performance in Simulated Marketscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  2. 2022HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models
  3. 2023Assessing the Impact of Prompting Methods on ChatGPT's Mathematical Capabilities
  4. 2023StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving
  5. 2023Large Language Models as Analogical Reasoners
  6. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  7. 2023The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models
  8. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models