mlsanityData to 7 Oct 2026

Leveraging Print Debugging to Improve Code Generation in Large Language Models

arXiv 2401.05319 ↗First posted 10 Jan 2024cs.CL cs.SEcited by 15

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have made significant progress in code generation tasks, but their performance in tackling programming problems with complex data structures and algorithms remains suboptimal. To address this issue, we propose an in-context learning approach that guides LLMs to debug by using a "print debugging" method, which involves inserting print statements to trace and analysing logs for fixing the bug. We collect a Leetcode problem dataset and evaluate our method using the Leetcode online judging system. Experiments with GPT-4 demonstrate the effectiveness of our approach, outperforming rubber duck debugging in easy and medium-level Leetcode problems by 1.5% and 17.9%.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 23 of its 24 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

23 of 24 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  11. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  12. 2023Teaching Large Language Models to Self-Debugcited by 963
11 more
  1. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  2. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  3. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  4. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  5. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 136
  6. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 110
  7. 2022Coder Reviewer Reranking for Code Generationcited by 83
  8. 2023SelfEvolve: A Code Evolution Framework via Large Language Modelscited by 43
  9. 2023Large Language Models Are Semi-Parametric Reinforcement Learning Agentscited by 42
  10. 2023Think Outside the Code: Brainstorming Boosts Large Language Models in Code Generationcited by 19
  11. 2023Explaining Competitive-Level Programming Solutions using LLMscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 5; 2026: 1.

15 papers.

Sort:

By area:

  1. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  2. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
  3. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  4. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  5. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 39
  6. 2024CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Executioncited by 31
  7. 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
  8. 2024ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generationcited by 14
  9. 2024Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Searchcited by 13
  10. 2024GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasetscited by 12
  11. 2024RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidancecited by 12
  12. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latencycited by 8
3 more
  1. 2025Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creationcited by 5
  2. 2025kAgent: An execution-guided crash resolution agent for the Linux kernelcited by 4
  3. 2026A Tool-Augmented, GPT-4 Chatbot for Real-Time Repository Data Analysiscited by 0