# CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models

arXiv: https://arxiv.org/abs/2206.00052 · HTML: https://mlsanity.com/paper/2206.00052/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 28 of 28)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Combating Adversarial Misspellings with Robust Word Recognition](https://mlsanity.com/paper/1905.11268/) (2019)
- [Technical Report on the CleverHans v2.1.0 Adversarial Examples Library](https://mlsanity.com/paper/1610.00768/) (2016)
- [An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation](https://mlsanity.com/paper/1812.08693/) (2018)
- [Natural Attack for Pre-trained Models of Code](https://mlsanity.com/paper/2201.08698/) (2022)
- [DOBF: A Deobfuscation Pre-Training Objective for Programming Languages](https://mlsanity.com/paper/2102.07492/) (2021)
- [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019)
- [Semantic Robustness of Models of Source Code](https://mlsanity.com/paper/2002.03043/) (2020)
- [Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data](https://mlsanity.com/paper/1805.12316/) (2018)
- [Adversarial Robustness of Deep Code Comment Generation](https://mlsanity.com/paper/2108.00213/) (2021)

## Built on by (most cited first, 30 of 53)

- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [ReCode: Robustness Evaluation of Code Generation Models](https://mlsanity.com/paper/2212.10264/) (2022)
- [Execution-based Code Generation using Deep Reinforcement Learning](https://mlsanity.com/paper/2301.13816/) (2023)
- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [Transformers Meet Directed Graphs](https://mlsanity.com/paper/2302.00049/) (2023)
- [Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks](https://mlsanity.com/paper/2308.04451/) (2023)
- [DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions](https://mlsanity.com/paper/2312.04730/) (2023)
- [Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps](https://mlsanity.com/paper/2309.04142/) (2023)
- [Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models](https://mlsanity.com/paper/2505.19616/) (2025)
- [Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates](https://mlsanity.com/paper/2402.05980/) (2024)
- [What can Large Language Models Capture about Code Functional Equivalence?](https://mlsanity.com/paper/2408.11081/) (2024)
- [Software Vulnerability and Functionality Assessment using LLMs](https://mlsanity.com/paper/2403.08429/) (2024)
- [Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities](https://mlsanity.com/paper/2212.10017/) (2022)
- [Token Alignment via Character Matching for Subword Completion](https://mlsanity.com/paper/2403.08688/) (2024)
- [FLRT: Fluent Student-Teacher Redteaming](https://mlsanity.com/paper/2407.17447/) (2024)
- [RedCoder: Automated Multi-Turn Red Teaming for Code LLMs](https://mlsanity.com/paper/2507.22063/) (2025)
- [RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents](https://mlsanity.com/paper/2510.02609/) (2025)
- [Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement](https://mlsanity.com/paper/2505.12060/) (2025)
- [Evolve the Method, Not the Prompts: Evolutionary Synthesis of Jailbreak Attacks on LLMs](https://mlsanity.com/paper/2511.12710/) (2025)
- [MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?](https://mlsanity.com/paper/2507.19598/) (2025)
- [NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks](https://mlsanity.com/paper/2510.03417/) (2025)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attack](https://mlsanity.com/paper/2601.20903/) (2026)
- [Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions](https://mlsanity.com/paper/2506.11111/) (2025)
- [A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection](https://mlsanity.com/paper/2512.16538/) (2025)
- [CodeFort: Robust Training for Code Generation Models](https://mlsanity.com/paper/2405.01567/) (2024)
- [XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants](https://mlsanity.com/paper/2503.14281/) (2025)
- [Evaluate-and-Purify: Fortifying Code Language Models Against Adversarial Attacks Using LLM-as-a-Judge](https://mlsanity.com/paper/2504.19730/) (2025)
- [MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models](https://mlsanity.com/paper/2509.14651/) (2025)
- [HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Models](https://mlsanity.com/paper/2510.18728/) (2025)

## Abstract (arXiv)

Pre-trained programming language (PL) models (such as CodeT5, CodeBERT, GraphCodeBERT, etc.,) have the potential to automate software engineering tasks involving code understanding and code generation. However, these models operate in the natural channel of code, i.e., they are primarily concerned with the human understanding of the code. They are not robust to changes in the input and thus, are potentially susceptible to adversarial attacks in the natural channel. We propose, CodeAttack, a simple yet effective black-box attack model that uses code structure to generate effective, efficient, and imperceptible adversarial code samples and demonstrates the vulnerabilities of the state-of-the-art PL models to code-specific adversarial attacks. We evaluate the transferability of CodeAttack on several code-code (translation and repair) and code-NL (summarization) tasks across different programming languages. CodeAttack outperforms state-of-the-art adversarial NLP attack models to achieve the best overall drop in performance while being more efficient, imperceptible, consistent, and fluent. The code can be found at https://github.com/reddy-lab-code-research/CodeAttack.