mlsanityData to 7 Oct 2026

Reasoning with Language Model Prompting: A Survey

arXiv 2212.09597 ↗First posted 19 Dec 2022cs.CL cs.AI cs.CVcited by 251

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Reasoning, as an essential ability for complex problem-solving, can provide back-end support for various real-world applications, such as medical diagnosis, negotiation, etc. This paper provides a comprehensive survey of cutting-edge research on reasoning with language model prompting. We introduce research works with comparisons and summaries and provide systematic resources to help beginners. We also discuss the potential reasons for emerging such reasoning abilities and highlight future research directions. Resources are available at https://github.com/zjunlp/Prompt4ReasoningPapers (updated periodically).

Key facts. Cited by 251 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 126 of its 218 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

126 of 218 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022Teaching Small Language Models to Reason16 Dec
  2. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning15 Dec
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  4. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  5. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  6. 2022Large Language Models are Zero-Shot Reasoners24 May
  7. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations24 May
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  10. 2022Iteratively Prompt Pre-trained Language Models for Chain of Thought16 Mar
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 115

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  10. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  11. 2018Hierarchical Neural Story Generationcited by 1,182
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  13. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  14. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  15. 2019How Can We Know What Language Models Know?cited by 953
  16. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  17. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  18. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  19. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  20. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  21. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  22. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  23. 2018Object Hallucination in Image Captioningcited by 661
  24. 2022Large Language Models Can Self-Improvecited by 656
  25. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  26. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  27. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  28. 2022PAL: Program-aided Language Modelscited by 635
  29. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  30. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  31. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  32. 2021Editing Factual Knowledge in Language Modelscited by 548
  33. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  34. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  35. 2021Fast Model Editing at Scalecited by 491
  36. 2022Memory-Based Model Editing at Scalecited by 443
  37. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  38. 2016Solving General Arithmetic Word Problemscited by 353

50 most cited of 115 All 115 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 81; 2024: 92; 2025: 65; 2026: 11.

251 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  4. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  5. 2022A Survey on In-context Learningcited by 717
  6. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  7. 2023Augmented Language Models: a Surveycited by 398
  8. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  9. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  10. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  11. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  12. 2024Tool Learning with Large Language Models: A Surveycited by 221
38 more
  1. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  2. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  3. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  4. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  5. 2024Symbolic Learning Enables Self-Evolving Agentscited by 104
  6. 2025LightThinker: Thinking Step-by-Step Compressioncited by 104
  7. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 100
  8. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
  9. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  10. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  11. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  12. 2025Memp: Exploring Agent Procedural Memorycited by 88
  13. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  14. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 86
  15. 2024Non-Determinism of "Deterministic" LLM Settingscited by 84
  16. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 82
  17. 2023LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representationscited by 81
  18. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  19. 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
  20. 2023Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)cited by 72
  21. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  22. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 71
  23. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  24. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  25. 2024Agent Planning with World Knowledge Modelcited by 66
  26. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  27. 2024Prompt Compression for Large Language Models: A Surveycited by 65
  28. 2023Making Large Language Models Better Reasoners with Alignmentcited by 63
  29. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  30. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63
  31. 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agentscited by 63
  32. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  33. 2023Natural Language Reasoning, A Surveycited by 61
  34. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agentscited by 61
  35. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  36. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  37. 2024Visual Agents as Fast and Slow Thinkerscited by 53
  38. 2023Data-Juicer: A One-Stop Data Processing System for Large Language Modelscited by 52

50 most cited of 251 All 251 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
  2. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  3. 2023Large Language Models as Analogical Reasoners
  4. 2025Navigating Semantic Relations: Challenges for Language Models in Abstract Common-Sense Reasoning
  5. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
  6. 2024Large Language Models are Contrastive Reasoners
  7. 2024Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models