# Natural Attack for Pre-trained Models of Code

arXiv: https://arxiv.org/abs/2201.08698 · HTML: https://mlsanity.com/paper/2201.08698/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2022.

## Built on by (most cited first, 30 of 75)

- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [ReCode: Robustness Evaluation of Code Generation Models](https://mlsanity.com/paper/2212.10264/) (2022)
- [CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models](https://mlsanity.com/paper/2206.00052/) (2022)
- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models](https://mlsanity.com/paper/2308.14401/) (2023)
- [Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection](https://mlsanity.com/paper/2306.17193/) (2023)
- [A DPLL(T) Framework for Verifying Deep Neural Networks](https://mlsanity.com/paper/2307.10266/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps](https://mlsanity.com/paper/2309.04142/) (2023)
- [Towards Fair Machine Learning Software: Understanding and Addressing Model Bias Through Counterfactual Thinking](https://mlsanity.com/paper/2302.08018/) (2023)
- [LiCoEval: Evaluating LLMs on License Compliance in Code Generation](https://mlsanity.com/paper/2408.02487/) (2024)
- [What can Large Language Models Capture about Code Functional Equivalence?](https://mlsanity.com/paper/2408.11081/) (2024)
- [Don't Complete It! Preventing Unhelpful Code Completion for Productive and Sustainable Neural Code Completion Systems](https://mlsanity.com/paper/2209.05948/) (2022)
- [An Exploratory Study on Code Attention in BERT](https://mlsanity.com/paper/2204.10200/) (2022)
- [Model Editing for LLMs4Code: How Far are We?](https://mlsanity.com/paper/2411.06638/) (2024)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code](https://mlsanity.com/paper/2312.14856/) (2023)
- [Token Alignment via Character Matching for Subword Completion](https://mlsanity.com/paper/2403.08688/) (2024)
- [An Extensive Study on Adversarial Attack against Pre-trained Models of Code](https://mlsanity.com/paper/2311.07553/) (2023)
- [PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models](https://mlsanity.com/paper/2401.15545/) (2024)
- [MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?](https://mlsanity.com/paper/2507.19598/) (2025)
- [LEAP: Efficient and Automated Test Method for NLP Software](https://mlsanity.com/paper/2308.11284/) (2023)
- [Boosting Source Code Learning with Text-Oriented Data Augmentation: An Empirical Study](https://mlsanity.com/paper/2303.06808/) (2023)
- [Towards Reliable Evaluation of Neural Program Repair with Natural Robustness Testing](https://mlsanity.com/paper/2402.11892/) (2024)
- [Data Quality Issues in Vulnerability Detection Datasets](https://mlsanity.com/paper/2410.06030/) (2024)
- [A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection](https://mlsanity.com/paper/2512.16538/) (2025)
- [Multi-Granularity Detector for Vulnerability Fixes](https://mlsanity.com/paper/2305.13884/) (2023)
- [CodeFort: Robust Training for Code Generation Models](https://mlsanity.com/paper/2405.01567/) (2024)

## Abstract (arXiv)

Pre-trained models of code have achieved success in many important software engineering tasks. However, these powerful models are vulnerable to adversarial attacks that slightly perturb model inputs to make a victim model produce wrong outputs. Current works mainly attack models of code with examples that preserve operational program semantics but ignore a fundamental requirement for adversarial example generation: perturbations should be natural to human judges, which we refer to as naturalness requirement. In this paper, we propose ALERT (nAturaLnEss AwaRe ATtack), a black-box attack that adversarially transforms inputs to make victim models produce wrong outputs. Different from prior works, this paper considers the natural semantic of generated examples at the same time as preserving the operational semantic of original inputs. Our user study demonstrates that human developers consistently consider that adversarial examples generated by ALERT are more natural than those generated by the state-of-the-art work by Zhang et al. that ignores the naturalness requirement. On attacking CodeBERT, our approach can achieve attack success rates of 53.62%, 27.79%, and 35.78% across three downstream tasks: vulnerability prediction, clone detection and code authorship attribution. On GraphCodeBERT, our approach can achieve average success rates of 76.95%, 7.96% and 61.47% on the three tasks. The above outperforms the baseline by 14.07% and 18.56% on the two pre-trained models on average. Finally, we investigated the value of the generated adversarial examples to harden victim models through an adversarial fine-tuning procedure and demonstrated the accuracy of CodeBERT and GraphCodeBERT against ALERT-generated adversarial examples increased by 87.59% and 92.32%, respectively.