# Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs

arXiv: https://arxiv.org/abs/2403.13271 · HTML: https://mlsanity.com/paper/2403.13271/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [TACO: Topics in Algorithmic COde generation dataset](https://mlsanity.com/paper/2312.14852/) (2023)
- [Self-planning Code Generation with Large Language Models](https://mlsanity.com/paper/2303.06689/) (2023)
- [Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data](https://mlsanity.com/paper/2302.12822/) (2023)
- [TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation](https://mlsanity.com/paper/1810.02720/) (2018)
- [Coder Reviewer Reranking for Code Generation](https://mlsanity.com/paper/2211.16490/) (2022)
- [Latent Predictor Networks for Code Generation](https://mlsanity.com/paper/1603.06744/) (2016)
- [Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generation](https://mlsanity.com/paper/2305.10679/) (2023)
- [IRCoCo: Immediate Rewards-Guided Deep Reinforcement Learning for Code Completion](https://mlsanity.com/paper/2401.16637/) (2024)
- [Knowledge-Aware Code Generation with Large Language Models](https://mlsanity.com/paper/2401.15940/) (2024)

## Built on by (most cited first, 17 of 17)

- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.26684/) (2026)
- [Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications](https://mlsanity.com/paper/2503.01245/) (2025)
- [CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code](https://mlsanity.com/paper/2404.15639/) (2024)
- [Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models](https://mlsanity.com/paper/2407.00456/) (2024)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulation](https://mlsanity.com/paper/2505.11604/) (2025)
- [AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners](https://mlsanity.com/paper/2505.16322/) (2025)
- [Eliminating Backdoors in Neural Code Models for Secure Code Understanding](https://mlsanity.com/paper/2408.04683/) (2024)
- [LawThinker: A Deep Research Legal Agent in Dynamic Environments](https://mlsanity.com/paper/2602.12056/) (2026)
- [Mastering the Craft of Data Synthesis for CodeLLMs](https://mlsanity.com/paper/2411.00005/) (2024)
- [Visual Program Distillation with Template-Based Augmentation](https://mlsanity.com/paper/2412.08564/) (2024)
- [ClaimDB: A Fact Verification Benchmark over Large Structured Data](https://mlsanity.com/paper/2601.14698/) (2026)
- [Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language](https://mlsanity.com/paper/2605.15607/) (2026)
- [Investigating the Transferability of Code Repair for Low-Resource Programming Languages](https://mlsanity.com/paper/2406.14867/) (2024)
- [Improving LLM Code Generation via Requirement-Aware Curriculum Reinforcement Learning](https://mlsanity.com/paper/2605.00433/) (2026)
- [ReAD: Reinforcement-Guided Capability Distillation for Large Language Models](https://mlsanity.com/paper/2605.11290/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have recently made significant advances in code generation through the 'Chain-of-Thought' prompting technique. This technique empowers the model to autonomously devise "solution plans" to tackle intricate programming challenges, thereby improving its performance in code generation. Nevertheless, smaller models have been struggling to keep up with LLMs in deducing these plans, adversely affecting their code generation capabilities. Given the considerable size and associated deployment costs, along with concerns about data security, many teams opt for deploying smaller models for code generation. Consequently, there arises a compelling need for transferring LLMs' code generation reasoning abilities to the smaller models. In this paper, we propose the CodePLAN framework, which aims to transfer LLMs' reasoning capabilities to smaller models through distillation. We adopt a multi-task learning approach, jointly undertaking code generation and solution plan generation tasks, to enhance the code generation capabilities of the smaller model. To ensure the superior quality of the solution plans, we advocate for the utilization of backward reasoning and plan sampling strategies. Our experiments show that in comparison to the conventional fine-tuning approach, our approach improves the smaller model's code generation performance (measured in pass@1 metric) by over 130% on the challenging APPS benchmark.