# Adversarial Robustness of Deep Code Comment Generation

arXiv: https://arxiv.org/abs/2108.00213 · HTML: https://mlsanity.com/paper/2108.00213/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2021.

## Built on by (most cited first, 22 of 22)

- [ReCode: Robustness Evaluation of Code Generation Models](https://mlsanity.com/paper/2212.10264/) (2022)
- [CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models](https://mlsanity.com/paper/2206.00052/) (2022)
- [DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions](https://mlsanity.com/paper/2312.04730/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variations](https://mlsanity.com/paper/2406.19783/) (2024)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [An Extensive Study on Adversarial Attack against Pre-trained Models of Code](https://mlsanity.com/paper/2311.07553/) (2023)
- [PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models](https://mlsanity.com/paper/2401.15545/) (2024)
- [ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization](https://mlsanity.com/paper/2407.01646/) (2024)
- [CodeFort: Robust Training for Code Generation Models](https://mlsanity.com/paper/2405.01567/) (2024)
- [XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants](https://mlsanity.com/paper/2503.14281/) (2025)
- [Evaluate-and-Purify: Fortifying Code Language Models Against Adversarial Attacks Using LLM-as-a-Judge](https://mlsanity.com/paper/2504.19730/) (2025)
- [Automatic Comment Generation via Multi-Pass Deliberation](https://mlsanity.com/paper/2209.06634/) (2022)
- [Enhancing Robustness of AI Offensive Code Generators via Data Augmentation](https://mlsanity.com/paper/2306.05079/) (2023)
- [CodeCipher: Learning to Obfuscate Source Code Against LLMs](https://mlsanity.com/paper/2410.05797/) (2024)
- [Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation](https://mlsanity.com/paper/2506.03535/) (2025)
- [Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing](https://mlsanity.com/paper/2309.05679/) (2023)
- [EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection](https://mlsanity.com/paper/2407.19216/) (2024)
- [Studying Vulnerable Code Entities in R](https://mlsanity.com/paper/2402.04421/) (2024)
- [A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair](https://mlsanity.com/paper/2604.21579/) (2026)
- [On the Lexical Superstition of Large Language Models for Code Comprehension: Re-evaluation on Code of Low Lexical Quality](https://mlsanity.com/paper/2609.26388/) (2026)

## Abstract (arXiv)

Deep neural networks (DNNs) have shown remarkable performance in a variety of domains such as computer vision, speech recognition, or natural language processing. Recently they also have been applied to various software engineering tasks, typically involving processing source code. DNNs are well-known to be vulnerable to adversarial examples, i.e., fabricated inputs that could lead to various misbehaviors of the DNN model while being perceived as benign by humans. In this paper, we focus on the code comment generation task in software engineering and study the robustness issue of the DNNs when they are applied to this task. We propose ACCENT, an identifier substitution approach to craft adversarial code snippets, which are syntactically correct and semantically close to the original code snippet, but may mislead the DNNs to produce completely irrelevant code comments. In order to improve the robustness, ACCENT also incorporates a novel training method, which can be applied to existing code comment generation models. We conduct comprehensive experiments to evaluate our approach by attacking the mainstream encoder-decoder architectures on two large-scale publicly available datasets. The results show that ACCENT efficiently produces stable attacks with functionality-preserving adversarial examples, and the generated examples have better transferability compared with baselines. We also confirm, via experiments, the effectiveness in improving model robustness with our training method.