# Poison Attack and Defense on Deep Source Code Processing Models

arXiv: https://arxiv.org/abs/2210.17029 · HTML: https://mlsanity.com/paper/2210.17029/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 32 of its 58 references that are in the map.
- First posted on arXiv 31 Oct 2022.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [ONION: A Simple and Effective Defense Against Textual Backdoor Attacks](https://mlsanity.com/paper/2011.10369/) (2020)
- [Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger](https://mlsanity.com/paper/2105.12400/) (2021)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair](https://mlsanity.com/paper/1901.01808/) (2018)
- [An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation](https://mlsanity.com/paper/1812.08693/) (2018)
- [Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases](https://mlsanity.com/paper/2007.15802/) (2020)
- [SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities](https://mlsanity.com/paper/1807.06756/) (2018)
- [Hidden Backdoors in Human-Centric Language Models](https://mlsanity.com/paper/2105.00164/) (2021)
- [Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution](https://mlsanity.com/paper/2106.06361/) (2021)
- [Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification](https://mlsanity.com/paper/2007.12070/) (2020)
- [CURE: Code-Aware Neural Machine Translation for Automatic Program Repair](https://mlsanity.com/paper/2103.00073/) (2021)
- [Rethinking the Trigger of Backdoor Attack](https://mlsanity.com/paper/2004.04692/) (2020)
- [Contrastive Code Representation Learning](https://mlsanity.com/paper/2007.04973/) (2020)
- [Detecting Code Clones with Graph Neural Networkand Flow-Augmented Abstract Syntax Tree](https://mlsanity.com/paper/2002.08653/) (2020)
- [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019)
- [A Syntax-Guided Edit Decoder for Neural Program Repair](https://mlsanity.com/paper/2106.08253/) (2021)
- [When Deep Learning Met Code Search](https://mlsanity.com/paper/1905.03813/) (2019)
- [CODIT: Code Editing with Tree-Based Neural Models](https://mlsanity.com/paper/1810.00314/) (2018)

## Built on by (most cited first, 11 of 11)

- [Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks](https://mlsanity.com/paper/2308.04451/) (2023)
- [Multi-target Backdoor Attacks for Code Pre-trained Models](https://mlsanity.com/paper/2306.08350/) (2023)
- [PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models](https://mlsanity.com/paper/2401.15545/) (2024)
- [A Systematic Review of Poisoning Attacks Against Large Language Models](https://mlsanity.com/paper/2506.06518/) (2025)
- [MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?](https://mlsanity.com/paper/2507.19598/) (2025)
- [Poisoning Programs by Un-Repairing Code: Security Concerns of AI-generated Code](https://mlsanity.com/paper/2403.06675/) (2024)
- [BadCS: A Backdoor Attack Framework for Code search](https://mlsanity.com/paper/2305.05503/) (2023)
- [Exploring the Security Threats of Retriever Backdoors in Retrieval-Augmented Code Generation](https://mlsanity.com/paper/2512.21681/) (2025)
- [FDI: Attack Neural Code Generation Systems through User Feedback Channel](https://mlsanity.com/paper/2408.04194/) (2024)
- [PDLRecover: Privacy-preserving Decentralized Model Recovery with Machine Unlearning](https://mlsanity.com/paper/2506.15112/) (2025)
- [Signature in Code Backdoor Detection, how far are we?](https://mlsanity.com/paper/2510.13992/) (2025)

## Abstract (arXiv)

In the software engineering community, deep learning (DL) has recently been applied to many source code processing tasks. Due to the poor interpretability of DL models, their security vulnerabilities require scrutiny. Recently, researchers have identified an emergent security threat, namely poison attack. The attackers aim to inject insidious backdoors into models by poisoning the training data with poison samples. Poisoned models work normally with clean inputs but produce targeted erroneous results with poisoned inputs embedded with triggers. By activating backdoors, attackers can manipulate the poisoned models in security-related scenarios. To verify the vulnerability of existing deep source code processing models to the poison attack, we present a poison attack framework for source code named CodePoisoner as a strong imaginary enemy. CodePoisoner can produce compilable even human-imperceptible poison samples and attack models by poisoning the training data with poison samples. To defend against the poison attack, we further propose an effective defense approach named CodeDetector to detect poison samples in the training data. CodeDetector can be applied to many model architectures and effectively defend against multiple poison attack approaches. We apply our CodePoisoner and CodeDetector to three tasks, including defect detection, clone detection, and code repair. The results show that (1) CodePoisoner achieves a high attack success rate (max: 100%) in misleading models to targeted erroneous behaviors. It validates that existing deep source code processing models have a strong vulnerability to the poison attack. (2) CodeDetector effectively defends against multiple poison attack approaches by detecting (max: 100%) poison samples in the training data. We hope this work can help practitioners notice the poison attack and inspire the design of more advanced defense techniques.