# CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules

arXiv: https://arxiv.org/abs/2310.08992 · HTML: https://mlsanity.com/paper/2310.08992/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 60 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 30 of 39)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning](https://mlsanity.com/paper/2207.01780/) (2022)
- [CodeT5+: Open Code Large Language Models for Code Understanding and Generation](https://mlsanity.com/paper/2305.07922/) (2023)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [RobustFill: Neural Program Learning under Noisy I/O](https://mlsanity.com/paper/1703.07469/) (2017)

## Built on by (most cited first, 30 of 58)

- [TextGrad: Automatic "Differentiation" via Text](https://mlsanity.com/paper/2406.07496/) (2024)
- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step](https://mlsanity.com/paper/2402.16906/) (2024)
- [Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering](https://mlsanity.com/paper/2401.08500/) (2024)
- [Agents in Software Engineering: Survey, Landscape, and Vision](https://mlsanity.com/paper/2409.09030/) (2024)
- [What's Wrong with Your Code Generated by Large Language Models? An Extensive Study](https://mlsanity.com/paper/2407.06153/) (2024)
- [Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs](https://mlsanity.com/paper/2404.15676/) (2024)
- [Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs](https://mlsanity.com/paper/2502.19411/) (2025)
- [AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation](https://mlsanity.com/paper/2312.13010/) (2023)
- [CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models](https://mlsanity.com/paper/2411.04329/) (2024)
- [CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation](https://mlsanity.com/paper/2501.07811/) (2025)
- [What Makes Large Language Models Reason in (Multi-Turn) Code Generation?](https://mlsanity.com/paper/2410.08105/) (2024)
- [LLM-Assisted Code Cleaning For Training Accurate Code Generators](https://mlsanity.com/paper/2311.14904/) (2023)
- [Large Language Model-based Data Science Agent: A Survey](https://mlsanity.com/paper/2508.02744/) (2025)
- [AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications](https://mlsanity.com/paper/2503.05346/) (2025)
- [Selective Prompt Anchoring for Code Generation](https://mlsanity.com/paper/2408.09121/) (2024)
- [Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification](https://mlsanity.com/paper/2603.26648/) (2026)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Problem Solving Through Human-AI Preference-Based Cooperation](https://mlsanity.com/paper/2408.07461/) (2024)
- [Measuring the Influence of Incorrect Code on Test Generation](https://mlsanity.com/paper/2409.09464/) (2024)
- [Multi-Programming Language Sandbox for LLMs](https://mlsanity.com/paper/2410.23074/) (2024)
- [Scattered Forest Search: Smarter Code Space Exploration with LLMs](https://mlsanity.com/paper/2411.05010/) (2024)
- [LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness](https://mlsanity.com/paper/2502.18489/) (2025)
- [PhysCodeBench: Benchmarking Physics-Aware Symbolic Simulation of 3D Scenes via Self-Corrective Multi-Agent Refinement](https://mlsanity.com/paper/2604.23580/) (2026)
- [MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks](https://mlsanity.com/paper/2312.15960/) (2023)
- [Large Language Models Synergize with Automated Machine Learning](https://mlsanity.com/paper/2405.03727/) (2024)
- [CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing](https://mlsanity.com/paper/2403.13583/) (2024)
- [INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness](https://mlsanity.com/paper/2407.02518/) (2024)
- [LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities](https://mlsanity.com/paper/2601.09822/) (2026)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) have already become quite proficient at solving simpler programming tasks like those in HumanEval or MBPP benchmarks. However, solving more complex and competitive programming tasks is still quite challenging for these models - possibly due to their tendency to generate solutions as monolithic code blocks instead of decomposing them into logical sub-tasks and sub-modules. On the other hand, experienced programmers instinctively write modularized code with abstraction for solving complex tasks, often reusing previously developed modules. To address this gap, we propose CodeChain, a novel framework for inference that elicits modularized code generation through a chain of self-revisions, each being guided by some representative sub-modules generated in previous iterations. Concretely, CodeChain first instructs the LLM to generate modularized codes through chain-of-thought prompting. Then it applies a chain of self-revisions by iterating the two steps: 1) extracting and clustering the generated sub-modules and selecting the cluster representatives as the more generic and re-usable implementations, and 2) augmenting the original chain-of-thought prompt with these selected module-implementations and instructing the LLM to re-generate new modularized solutions. We find that by naturally encouraging the LLM to reuse the previously developed and verified sub-modules, CodeChain can significantly boost both modularity as well as correctness of the generated solutions, achieving relative pass@1 improvements of 35% on APPS and 76% on CodeContests. It is shown to be effective on both OpenAI LLMs as well as open-sourced LLMs like WizardCoder. We also conduct comprehensive ablation studies with different methods of prompting, number of clusters, model sizes, program qualities, etc., to provide useful insights that underpin CodeChain's success.