# What Makes Large Language Models Reason in (Multi-Turn) Code Generation?

arXiv: https://arxiv.org/abs/2410.08105 · HTML: https://mlsanity.com/paper/2410.08105/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 48 of its 53 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 30 of 48)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [Scaling Relationship on Learning Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2308.01825/) (2023)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Iterative Reasoning Preference Optimization](https://mlsanity.com/paper/2404.19733/) (2024)
- [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024)

## Built on by (most cited first, 29 of 29)

- [Optimizing Language Models for Inference Time Objectives using Reinforcement Learning](https://mlsanity.com/paper/2503.19595/) (2025)
- [Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMs](https://mlsanity.com/paper/2502.19411/) (2025)
- [Teaching Language Models to Critique via Reinforcement Learning](https://mlsanity.com/paper/2502.03492/) (2025)
- [Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation](https://mlsanity.com/paper/2412.15118/) (2024)
- [A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks](https://mlsanity.com/paper/2501.10069/) (2025)
- [debug-gym: A Text-Based Environment for Interactive Debugging](https://mlsanity.com/paper/2503.21557/) (2025)
- [MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference](https://mlsanity.com/paper/2510.07475/) (2025)
- [Towards Spoken Mathematical Reasoning: Benchmarking Speech-based Models over Multi-faceted Math Problems](https://mlsanity.com/paper/2505.15000/) (2025)
- [Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers](https://mlsanity.com/paper/2505.20128/) (2025)
- [RedCoder: Automated Multi-Turn Red Teaming for Code LLMs](https://mlsanity.com/paper/2507.22063/) (2025)
- [Improving Diversity in Language Models: When Temperature Fails, Change the Loss](https://mlsanity.com/paper/2508.09654/) (2025)
- [Mitigating Conversational Inertia in Multi-Turn Agents](https://mlsanity.com/paper/2602.03664/) (2026)
- [Large Language Models for Interpretable Mental Health Diagnosis](https://mlsanity.com/paper/2501.07653/) (2025)
- [The KoLMogorov Test: Compression by Code Generation](https://mlsanity.com/paper/2503.13992/) (2025)
- [FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration](https://mlsanity.com/paper/2510.04852/) (2025)
- [Self-Correcting Code Generation Using Small Language Models](https://mlsanity.com/paper/2505.23060/) (2025)
- [Benchmarking Correctness and Security in Multi-Turn Code Generation](https://mlsanity.com/paper/2510.13859/) (2025)
- [Falsification, Not Exposure: An Internally Preregistered Placebo-Controlled Decomposition of Self-Repair Feedback in Frozen Small Code Models](https://mlsanity.com/paper/2606.31511/) (2026)
- [Self-Execution Simulation Improves Coding Models](https://mlsanity.com/paper/2604.03253/) (2026)
- [Evolutionary Pre-Prompt Optimization for Mathematical Reasoning](https://mlsanity.com/paper/2412.04291/) (2024)
- [Intention Chain-of-Thought Prompting with Dynamic Routing for Code Generation](https://mlsanity.com/paper/2512.14048/) (2025)
- [Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training](https://mlsanity.com/paper/2507.01752/) (2025)
- [Breaking the Myth: Can Small Models Infer Postconditions Too?](https://mlsanity.com/paper/2507.10182/) (2025)
- [SELF-REDRAFT: Eliciting Intrinsic Exploration-Exploitation Balance in Test-Time Scaling for Code Generation](https://mlsanity.com/paper/2511.02854/) (2025)
- [RPM-MCTS: Knowledge-Retrieval as Process Reward Model with Monte Carlo Tree Search for Code Generation](https://mlsanity.com/paper/2511.19895/) (2025)
- [Towards a Neural Debugger for Python](https://mlsanity.com/paper/2603.09951/) (2026)
- [Beyond pass@k: Redundancy-Aware RLVR for Multi-Sample Code Generation](https://mlsanity.com/paper/2605.28022/) (2026)
- [SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs](https://mlsanity.com/paper/2506.11081/) (2025)
- [code_transformed: The Influence of Large Language Models on Code](https://mlsanity.com/paper/2506.12014/) (2025)

## Abstract (arXiv)

Prompting techniques such as chain-of-thought have established themselves as a popular vehicle for improving the outputs of large language models (LLMs). For code generation, however, their exact mechanics and efficacy are under-explored. We thus investigate the effects of a wide range of prompting strategies with a focus on automatic re-prompting over multiple turns and computational requirements. After systematically decomposing reasoning, instruction, and execution feedback prompts, we conduct an extensive grid search on the competitive programming benchmarks CodeContests and TACO for multiple LLM families and sizes (Llama 3.0 and 3.1, 8B, 70B, 405B, and GPT-4o). Our study reveals strategies that consistently improve performance across all models with small and large sampling budgets. We then show how finetuning with such an optimal configuration allows models to internalize the induced reasoning process and obtain improvements in performance and scalability for multi-turn code generation.