# Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges

arXiv: https://arxiv.org/abs/2002.05442 · HTML: https://mlsanity.com/paper/2002.05442/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2020.

## Built on by (most cited first, 30 of 55)

- [Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering](https://mlsanity.com/paper/2403.03163/) (2024)
- [Large Language Models Meet NL2Code: A Survey](https://mlsanity.com/paper/2212.09420/) (2022)
- [SALLM: Security Assessment of Generated Code](https://mlsanity.com/paper/2311.00889/) (2023)
- [Adaptive In-conversation Team Building for Language Model Agents](https://mlsanity.com/paper/2405.19425/) (2024)
- [Execution-based Evaluation for Data Science Code Generation Models](https://mlsanity.com/paper/2211.09374/) (2022)
- [Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs](https://mlsanity.com/paper/2103.09499/) (2021)
- [Large Language Models of Code Fail at Completing Code with Potential Bugs](https://mlsanity.com/paper/2306.03438/) (2023)
- [Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping](https://mlsanity.com/paper/2410.16232/) (2024)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting](https://mlsanity.com/paper/2103.07845/) (2021)
- [Memorization and Generalization in Neural Code Intelligence Models](https://mlsanity.com/paper/2106.08704/) (2021)
- [PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)](https://mlsanity.com/paper/2409.12699/) (2024)
- [A Survey on Data-driven Software Vulnerability Assessment and Prioritization](https://mlsanity.com/paper/2107.08364/) (2021)
- [Assemble Foundation Models for Automatic Code Summarization](https://mlsanity.com/paper/2201.05222/) (2022)
- [FRANC: A Lightweight Framework for High-Quality Code Generation](https://mlsanity.com/paper/2307.08220/) (2023)
- [DeepCVA: Automated Commit-level Vulnerability Assessment with Deep Multi-task Learning](https://mlsanity.com/paper/2108.08041/) (2021)
- [Leveraging Unsupervised Learning to Summarize APIs Discussed in Stack Overflow](https://mlsanity.com/paper/2111.13962/) (2021)
- [Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study](https://mlsanity.com/paper/2401.11105/) (2024)
- [The Fault in our Stars: Quality Assessment of Code Generation Benchmarks](https://mlsanity.com/paper/2404.10155/) (2024)
- [Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT](https://mlsanity.com/paper/2404.17110/) (2024)
- [LLMSecConfig: An LLM-Based Approach for Fixing Software Container Misconfigurations](https://mlsanity.com/paper/2502.02009/) (2025)
- [T5QL: Taming language models for SQL generation](https://mlsanity.com/paper/2209.10254/) (2022)
- [Automating Code-Related Tasks Through Transformers: The Impact of Pre-training](https://mlsanity.com/paper/2302.04048/) (2023)
- [Machine Learning for Software Engineering: A Tertiary Study](https://mlsanity.com/paper/2211.09425/) (2022)
- [Automated Security Assessment for the Internet of Things](https://mlsanity.com/paper/2109.04029/) (2021)
- [LEAP: Efficient and Automated Test Method for NLP Software](https://mlsanity.com/paper/2308.11284/) (2023)
- [Mitigating Data Imbalance for Software Vulnerability Assessment: Does Data Augmentation Help?](https://mlsanity.com/paper/2407.10722/) (2024)
- [Automated Code-centric Software Vulnerability Assessment: How Far Are We? An Empirical Study in C/C++](https://mlsanity.com/paper/2407.17053/) (2024)
- [Automatic Data Labeling for Software Vulnerability Prediction Models: How Far Are We?](https://mlsanity.com/paper/2407.17803/) (2024)
- [On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models](https://mlsanity.com/paper/2203.08417/) (2022)

## Abstract (arXiv)

Deep Learning (DL) techniques for Natural Language Processing have been evolving remarkably fast. Recently, the DL advances in language modeling, machine translation and paragraph understanding are so prominent that the potential of DL in Software Engineering cannot be overlooked, especially in the field of program learning. To facilitate further research and applications of DL in this field, we provide a comprehensive review to categorize and investigate existing DL methods for source code modeling and generation. To address the limitations of the traditional source code models, we formulate common program learning tasks under an encoder-decoder framework. After that, we introduce recent DL mechanisms suitable to solve such problems. Then, we present the state-of-the-art practices and discuss their challenges with some recommendations for practitioners and researchers as well.