mlsanityData to 7 Oct 2026

Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models

arXiv 2310.03965 ↗First posted 6 Oct 2023cs.AI cs.CLcited by 37

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large Language Models (LLMs) have achieved remarkable success in reasoning tasks with the development of prompting methods. However, existing prompting approaches cannot reuse insights of solving similar problems and suffer from accumulated errors in multi-step reasoning, since they prompt LLMs to reason textitfrom scratch. To address these issues, we propose textbftextitThought Propagation (TP), which explores the analogous problems and leverages their solutions to enhance the complex reasoning ability of LLMs. These analogous problems are related to the input one, with reusable solutions and problem-solving strategies. Thus, it is promising to propagate insights of solving previous analogous problems to inspire new problem-solving. To achieve this, TP first prompts LLMs to propose and solve a set of analogous problems that are related to the input one. Then, TP reuses the results of analogous problems to directly yield a new solution or derive a knowledge-intensive plan for execution to amend the initial solution obtained from scratch. TP is compatible with existing prompting approaches, allowing plug-and-play generalization and enhancement in a wide range of tasks without much labor in task-specific prompt engineering. Experiments across three challenging tasks demonstrate TP enjoys a substantial improvement over the baselines by an average of 12% absolute increase in finding the optimal solutions in Shortest-path Reasoning, 13% improvement of human preference in Creative Writing, and 15% enhancement in the task completion rate of LLM-Agent Planning.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 65 of its 74 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

65 of 74 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  2. 2022Reasoning with Language Model Prompting: A Survey19 Dec
  3. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  4. 2022Large Language Models are Zero-Shot Reasoners24 May
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 58

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
38 more
  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  5. 2023Teaching Large Language Models to Self-Debugcited by 963
  6. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  9. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  10. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  11. 2022Large Language Models Can Self-Improvecited by 656
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  13. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  14. 2023Augmented Language Models: a Surveycited by 398
  15. 2022Self-critiquing models for assisting human evaluatorscited by 372
  16. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  17. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  18. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  19. 2023Large Language Models as Tool Makerscited by 293
  20. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  21. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  22. 2022Generating Sequences by Learning to Self-Correctcited by 271
  23. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  24. 2023How is ChatGPT's behavior changing over time?cited by 265
  25. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  26. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  27. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  28. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  29. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  30. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  31. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  32. 2020Graph Information Bottleneck for Subgraph Recognitioncited by 119
  33. 2023Shepherd: A Critic for Language Model Generationcited by 96
  34. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  35. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  36. 2021SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanismcited by 75
  37. 2022Retrieval-based Controllable Molecule Generationcited by 44
  38. 2023Can Large Language Models Empower Molecular Property Prediction?cited by 43

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 18; 2025: 9; 2026: 8.

37 papers.

Sort:

By area:

  1. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  4. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  5. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
  6. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  7. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  8. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  9. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 55
  10. 2025Theoretical Benefit and Limitation of Diffusion Language Modelcited by 46
  11. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  12. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 38
25 more
  1. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 29
  2. 2024Meta Reasoning for Large Language Modelscited by 29
  3. 2023Mathematical Language Models: A Surveycited by 24
  4. 2024Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoningcited by 24
  5. 2026Large Language Model Reasoning Failurescited by 20
  6. 2025AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Searchcited by 14
  7. 2024Toward Adaptive Reasoning in Large Language Models with Thought Rollbackcited by 11
  8. 2025Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agentscited by 11
  9. 2025Instantiation-based Formalization of Logical Reasoning Tasks using Language Models and Logical Solverscited by 7
  10. 2024Large language model empowered participatory urban planningcited by 5
  11. 2024Enhancing Collaborative Semantics of Language Model-Driven Recommendations via Graph-Aware Learningcited by 5
  12. 2026MASPOB: Bandit-Based Prompt Optimization for Multi-Agent Systems with Graph Neural Networkscited by 5
  13. 2024A Study on Training and Developing Large Language Models for Behavior Tree Generationcited by 4
  14. 2024DeFine: Decision-Making with Analogical Reasoning over Factor Profilescited by 3
  15. 2025Vision-and-Language Navigation with Analogical Textual Descriptions in LLMscited by 3
  16. 2026FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoningcited by 3
  17. 2024A Survey on the Integration of Generative AI for Critical Thinking in Mobile Networkscited by 1
  18. 2025Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compressioncited by 1
  19. 2026Can Large Language Models Generalize Procedures Across Representations?cited by 1
  20. 2024Prompt Recursive Search: A Living Framework with Adaptive Growth in LLM Auto-Promptingcited by 0
  21. 2025Universe of Thoughts: Enabling Creative Reasoning with Large Language Modelscited by 0
  22. 2026Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoningcited by 0
  23. 2026Topology of Reasoning: Retrieved Cell Complex-Augmented Generation for Textual Graph Question Answeringcited by 0
  24. 2026Structural Rationale Distillation via Reasoning Space Compressioncited by 0
  25. 2026Analogical Deep Research: Retrieving and Integrating Historical Analogies for Foresight Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models
  2. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought
  3. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  4. 2023Large Language Models as Analogical Reasoners
  5. 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
  6. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  7. 2025Effectively Controlling Reasoning Models through Thinking Intervention
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models