mlsanityData to 7 Oct 2026

Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions

arXiv 2212.10561 ↗First posted 20 Dec 2022cs.CL cs.AI cs.LGcited by 62

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Despite recent success in large language model (LLM) reasoning, LLMs struggle with hierarchical multi-step reasoning tasks like generating complex programs. For these tasks, humans often start with a high-level algorithmic design and implement each part gradually. We introduce Parsel, a framework enabling automatic implementation and validation of complex algorithms with code LLMs. With Parsel, we automatically decompose algorithmic tasks into hierarchical natural language function descriptions and then search over combinations of possible function implementations using tests. We show that Parsel can be used across domains requiring hierarchical reasoning, including program synthesis and robotic planning. We find that, using Parsel, LLMs solve more competition-level problems in the APPS dataset, resulting in pass rates over 75% higher than prior results from directly sampling AlphaCode and Codex, while often using a smaller sample budget. Moreover, with automatically generated tests, we find that Parsel can improve the state-of-the-art pass@1 performance on HumanEval from 67% to 85%. We also find that LLM-generated robotic plans using Parsel are more than twice as likely to be considered accurate than directly generated plans. Lastly, we explore how Parsel addresses LLM limitations and discuss how Parsel may be useful for human programmers. We release our code at https://github.com/ezelikman/parsel

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 49 of its 76 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

49 of 76 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct
  2. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  11. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  12. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
35 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 879
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  3. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  4. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  5. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  6. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  7. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  8. 2022CodeT: Code Generation with Generated Testscited by 369
  9. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  10. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  11. 2022Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofscited by 278
  12. 2022Can language models learn from explanations in context?cited by 248
  13. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 245
  14. 2022Binding Language Models in Symbolic Languagescited by 224
  15. 2022Exploring Length Generalization in Large Language Modelscited by 204
  16. 2022Do Users Write More Insecure Code with AI Assistants?cited by 186
  17. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  18. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  19. 2022Multi-lingual Evaluation of Code Generation Modelscited by 158
  20. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  21. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 111
  22. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  23. 2022Language Model Cascadescited by 104
  24. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  25. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 86
  26. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  27. 2022Coder Reviewer Reranking for Code Generationcited by 83
  28. 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 81
  29. 2022A Generalist Neural Algorithmic Learnercited by 58
  30. 2022Top-Down Synthesis for Library Learningcited by 57
  31. 2021Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?cited by 55
  32. 2015Program Synthesis using Natural Languagecited by 41
  33. 2022Sub-Task Decomposition Enables Learning in Sequence to Sequence Taskscited by 41
  34. 2021Communicating Natural Programs to Humans and Machinescited by 35
  35. 2020BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Explorationcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 32; 2025: 14; 2026: 8.

62 papers.

Sort:

By area:

  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  2. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  3. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  4. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  5. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  6. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  7. 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 89
  8. 2024Symbolic Regression with a Learned Concept Librarycited by 76
  9. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  10. 2025A Self-Improving Coding Agentcited by 58
  11. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 50
  12. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 46
38 more
  1. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  2. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  3. 2025Repo2Run: Automated Building Executable Environment for Code Repository at Scalecited by 39
  4. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
  5. 2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
  6. 2023AI capabilities can be significantly improved without expensive retrainingcited by 28
  7. 2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28
  8. 2024Is Programming by Example solved by LLMs?cited by 27
  9. 2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Webcited by 23
  10. 2025Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematicscited by 22
  11. 2025Large Language Model-based Data Science Agent: A Surveycited by 22
  12. 2023Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generationcited by 19
  13. 2024Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generationcited by 19
  14. 2024Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysiscited by 16
  15. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  16. 2023Evaluating statistical language models as pragmatic reasonerscited by 14
  17. 2024Learning Task Decomposition to Assist Humans in Competitive Programmingcited by 14
  18. 2024NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalizationcited by 13
  19. 2025SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPscited by 11
  20. 2025Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approachcited by 10
  21. 2024"Set It Up!": Functional Object Arrangement with Compositional Generative Modelscited by 9
  22. 2024FactorSim: Generative Simulation via Factorized Representationcited by 9
  23. 2024FVEval: Understanding Language Model Capabilities in Formal Verification of Digital Hardwarecited by 9
  24. 2025DISC: Dynamic Decomposition Improves LLM Inference Scalingcited by 9
  25. 2024PLUM: Improving Code LMs with Execution-Guided On-Policy Preference Learning Driven By Synthetic Test Casescited by 8
  26. 2026Understanding the Challenges in Iterative Generative Optimization with LLMscited by 8
  27. 2024Planning-Driven Programming: A Large Language Model Programming Workflowcited by 7
  28. 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunitiescited by 7
  29. 2024DCA-Bench: A Benchmark for Dataset Curation Agentscited by 6
  30. 2025CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generationcited by 6
  31. 2025Understanding and Enhancing the Planning Capability of Language Models via Multi-Token Predictioncited by 6
  32. 2023AskIt: Unified Programming Interface for Programming with Large Language Modelscited by 5
  33. 2025Distinct Computations Emerge From Compositional Curricula in In-Context Learningcited by 5
  34. 2024Understanding How CodeLLMs (Mis)Predict Types with Activation Steeringcited by 4
  35. 2024Testing the Effect of Code Documentation on Large Language Model Code Understandingcited by 4
  36. 2024Towards Learning Foundation Models for Heuristic Functions to Solve Pathfinding Problemscited by 4
  37. 2024MathLearner: A Large Language Model Agent Framework for Learning to Solve Mathematical Problemscited by 4
  38. 2026Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalabilitycited by 4

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Compute-Accuracy Pareto Frontiers for Open-Source Reasoning Large Language Models
  2. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
  3. 2025CRISP: Complex Reasoning with Interpretable Step-based Plans
  4. 2025From Implicit Exploration to Structured Reasoning: Leveraging Guideline and Refinement for LLMs
  5. 2025Implicit Reasoning in Large Language Models: A Comprehensive Survey
  6. 2025SLR: Automated Synthesis for Scalable Logical Reasoning
  7. 2025Code Simulation as a Proxy for High-order Tasks in Large Language Models
  8. 2024ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents