# Source Code Data Augmentation for Deep Learning: A Survey

arXiv: https://arxiv.org/abs/2305.19915 · HTML: https://mlsanity.com/paper/2305.19915/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 123 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [SantaCoder: don't reach for the stars!](https://mlsanity.com/paper/2301.03988/) (2023)
- [Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity](https://mlsanity.com/paper/2301.12867/) (2023)
- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [ICE-Score: Instructing Large Language Models to Evaluate Code](https://mlsanity.com/paper/2304.14317/) (2023)
- [Measuring The Impact Of Programming Language Distribution](https://mlsanity.com/paper/2302.01973/) (2023)
- [How Important are Good Method Names in Neural Code Generation? A Model Robustness Perspective](https://mlsanity.com/paper/2211.15844/) (2022)
- [CodeS: Towards Code Model Generalization Under Distribution Shift](https://mlsanity.com/paper/2206.05480/) (2022)
- [Pre-Training Representations of Binary Code Using Contrastive Learning](https://mlsanity.com/paper/2210.05102/) (2022)
- [STRATA: Simple, Gradient-Free Attacks for Models of Code](https://mlsanity.com/paper/2009.13562/) (2020)
- [SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models](https://mlsanity.com/paper/2210.04802/) (2022)
- [Boosting Source Code Learning with Text-Oriented Data Augmentation: An Empirical Study](https://mlsanity.com/paper/2303.06808/) (2023)
- [AugmentedCode: Examining the Effects of Natural Language Resources in Code Retrieval Models](https://mlsanity.com/paper/2110.08512/) (2021)
- [Evaluation of Contrastive Learning with Various Code Representations for Code Clone Detection](https://mlsanity.com/paper/2206.08726/) (2022)
- [Pathways to Leverage Transcompiler based Data Augmentation for Cross-Language Clone Detection](https://mlsanity.com/paper/2303.01435/) (2023)

## Built on by (most cited first, 3 of 3)

- [EffiBench: Benchmarking the Efficiency of Automatically Generated Code](https://mlsanity.com/paper/2402.02037/) (2024)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [LLM-based Vulnerable Code Augmentation: Generate or Refactor?](https://mlsanity.com/paper/2512.08493/) (2025)

## Abstract (arXiv)

The increasingly popular adoption of deep learning models in many critical source code tasks motivates the development of data augmentation (DA) techniques to enhance training data and improve various capabilities (e.g., robustness and generalizability) of these models. Although a series of DA methods have been proposed and tailored for source code models, there lacks a comprehensive survey and examination to understand their effectiveness and implications. This paper fills this gap by conducting a comprehensive and integrative survey of data augmentation for source code, wherein we systematically compile and encapsulate existing literature to provide a comprehensive overview of the field. We start with an introduction of data augmentation in source code and then provide a discussion on major representative approaches. Next, we highlight the general strategies and techniques to optimize the DA quality. Subsequently, we underscore techniques useful in real-world source code scenarios and downstream tasks. Finally, we outline the prevailing challenges and potential opportunities for future research. In essence, we aim to demystify the corpus of existing literature on source code DA for deep learning, and foster further exploration in this sphere. Complementing this, we present a continually updated GitHub repository that hosts a list of update-to-date papers on DA for source code modeling, accessible at urlhttps://github.com/terryyz/DataAug4Code.