mlsanityData to 7 Oct 2026

Decomposed Prompting: A Modular Approach for Solving Complex Tasks

arXiv 2210.02406 ↗First posted 5 Oct 2022cs.CLcited by 554

Not on the map. Why?

Cited by 208 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Few-shot prompting is a surprisingly powerful way to use Large Language Models (LLMs) to solve various tasks. However, this approach struggles as the task complexity increases or when the individual reasoning steps of the task themselves are hard to learn, especially when embedded in more complex tasks. To address this, we propose Decomposed Prompting, a new approach to solve complex tasks by decomposing them (via prompting) into simpler sub-tasks that can be delegated to a library of prompting-based LLMs dedicated to these sub-tasks. This modular structure allows each prompt to be optimized for its specific sub-task, further decomposed if necessary, and even easily replaced with more effective prompts, trained models, or symbolic functions if desired. We show that the flexibility and modularity of Decomposed Prompting allows it to outperform prior work on few-shot prompting using GPT3. On symbolic reasoning tasks, we can further decompose sub-tasks that are hard for LLMs into even simpler solvable sub-tasks. When the complexity comes from the input length, we can recursively decompose the task into the same task but with smaller inputs. We also evaluate our approach on textual multi-step reasoning tasks: on long-context multi-hop QA task, we can more effectively teach the sub-tasks via our separate sub-tasks prompts; and on open-domain multi-hop QA, we can incorporate a symbolic information retrieval within our decomposition framework, leading to improved performance on both tasks. Datasets, Code and Prompts available at https://github.com/allenai/DecomP.

Key facts. Cited by 554 papers in the map (data through 7 Oct 2026). Builds on 33 of its 114 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

33 of 114 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  11. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  12. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
21 more
  1. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  2. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  3. 2015Neural Module Networkscited by 375
  4. 2016Solving General Arithmetic Word Problemscited by 353
  5. 2021True Few-Shot Learning with Language Modelscited by 343
  6. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  7. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  8. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  9. 2019Multi-hop Reading Comprehension through Question Decomposition and Rescoringcited by 119
  10. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  11. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  12. 2022Language Model Cascadescited by 104
  13. 2015Neural Programmer-Interpreterscited by 96
  14. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  15. 2019Neural Module Networks for Reasoning over Textcited by 65
  16. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  17. 2017Making Neural Programming Architectures Generalize via Recursioncited by 35
  18. 2022Standing on the Shoulders of Giant Frozen Language Modelscited by 33
  19. 2015Neural Programmer: Inducing Latent Programs with Gradient Descentcited by 32
  20. 2019Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoningcited by 28
  21. 2021Hey AI, Can You Solve Complex Tasks by Talking to Agents?cited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 124; 2024: 134; 2025: 163; 2026: 121.

554 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  3. 2023Teaching Large Language Models to Self-Debugcited by 963
  4. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  5. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  6. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  7. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  8. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  9. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  10. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  11. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  12. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
38 more
  1. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  2. 2023Active Retrieval Augmented Generationcited by 506
  3. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  4. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  5. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  6. 2023Augmented Language Models: a Surveycited by 398
  7. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  8. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  9. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
  10. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  12. 2023Tool Learning with Foundation Modelscited by 302
  13. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  14. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  15. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  16. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  17. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  18. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  19. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  20. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  21. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  22. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 193
  23. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
  24. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  25. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 179
  26. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  27. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 172
  28. 2023An LLM Compiler for Parallel Function Callingcited by 170
  29. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  30. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 157
  31. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solverscited by 155
  32. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?cited by 153
  33. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 137
  34. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  35. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  36. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  37. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  38. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 124

50 most cited of 554 All 554 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Complexity-Based Prompting for Multi-Step Reasoning
  2. 2024Towards Generalist Prompting for Large Language Models by Mental Models
  3. 2025Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs
  4. 2026Diverge to Induce Prompting: Multi-Rationale Induction for Zero-Shot Reasoning
  5. 2026Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs
  6. 2026Beyond the Prompt in Large Language Models: Comprehension, In-Context Learning, and Chain-of-Thought
  7. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  8. 2022Large Language Models are Zero-Shot Reasoners