# A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair

arXiv: https://arxiv.org/abs/2304.08191 · HTML: https://mlsanity.com/paper/2304.08191/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2023.

## Built on by (most cited first, 26 of 26)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair](https://mlsanity.com/paper/2309.00608/) (2023)
- [A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification](https://mlsanity.com/paper/2305.14752/) (2023)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Automatic Code Summarization via ChatGPT: How Far Are We?](https://mlsanity.com/paper/2305.12865/) (2023)
- [Log Parsing: How Far Can ChatGPT Go?](https://mlsanity.com/paper/2306.01590/) (2023)
- [CigaR: Cost-efficient Program Repair with LLMs](https://mlsanity.com/paper/2402.06598/) (2024)
- [An Assessment of ChatGPT on Log Data](https://mlsanity.com/paper/2309.07938/) (2023)
- [LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language](https://mlsanity.com/paper/2402.16929/) (2024)
- [ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning](https://mlsanity.com/paper/2310.05143/) (2023)
- [CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking](https://mlsanity.com/paper/2402.11842/) (2024)
- [Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineering](https://mlsanity.com/paper/2411.15501/) (2024)
- [ConDefects: A New Dataset to Address the Data Leakage Concern for LLM-based Fault Localization and Program Repair](https://mlsanity.com/paper/2310.16253/) (2023)
- [Evaluating the Performance of ChatGPT for Spam Email Detection](https://mlsanity.com/paper/2402.15537/) (2024)
- [SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework](https://mlsanity.com/paper/2411.19234/) (2024)
- [Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency](https://mlsanity.com/paper/2504.15989/) (2025)
- [FidelityGPT: Correcting Decompilation Distortions with Retrieval Augmented Generation](https://mlsanity.com/paper/2510.19615/) (2025)
- [Exploring Large Language Models in Resolving Environment-Related Crash Bugs: Localizing and Repairing](https://mlsanity.com/paper/2312.10448/) (2023)
- [Beyond Keywords: A Context-based Hybrid Approach to Mining Ethical Concern-related App Reviews](https://mlsanity.com/paper/2411.07398/) (2024)
- [SIExVulTS: Sensitive Information Exposure Vulnerability Detection System using Transformer Models and Static Analysis](https://mlsanity.com/paper/2508.19472/) (2025)
- [Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignments](https://mlsanity.com/paper/2404.01754/) (2024)
- [Large model retrieval enhancement framework for construction site risk identification](https://mlsanity.com/paper/2508.02073/) (2025)
- [Improving MPI Error Detection and Repair with Large Language Models and Bug References](https://mlsanity.com/paper/2604.02398/) (2026)
- [Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality](https://mlsanity.com/paper/2607.14816/) (2026)
- [MultiFixer: A Coordinator-Proposer Based Multi-Agent Framework For Fixing Multi-Hunk Bugs](https://mlsanity.com/paper/2607.26591/) (2026)

## Abstract (arXiv)

ChatGPT has revolutionized many research and industrial fields. ChatGPT has shown great potential in software engineering to boost various traditional tasks such as program repair, code understanding, and code generation. However, whether automatic program repair (APR) applies to deep learning (DL) programs is still unknown. DL programs, whose decision logic is not explicitly encoded in the source code, have posed unique challenges to APR. While to repair DL programs, an APR approach needs to not only parse the source code syntactically but also needs to understand the code intention. With the best prior work, the performance of fault localization is still far less than satisfactory (only about 30%). Therefore, in this paper, we explore ChatGPT's capability for DL program repair by asking three research questions. (1) Can ChatGPT debug DL programs effectively? (2) How can ChatGPT's repair performance be improved by prompting? (3) In which way can dialogue help facilitate the repair? On top of that, we categorize the common aspects useful for prompt design for DL program repair. Also, we propose various prompt templates to facilitate the performance and summarize the advantages and disadvantages of ChatGPT's abilities such as detecting bad code smell, code refactoring, and detecting API misuse/deprecation.