# Poisoning Programs by Un-Repairing Code: Security Concerns of AI-generated Code

arXiv: https://arxiv.org/abs/2403.06675 · HTML: https://mlsanity.com/paper/2403.06675/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 12 of 12)

- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks](https://mlsanity.com/paper/1804.00792/) (2018)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [Hidden Backdoors in Human-Centric Language Models](https://mlsanity.com/paper/2105.00164/) (2021)
- [Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks](https://mlsanity.com/paper/2102.02017/) (2021)
- [Backdoors in Neural Models of Source Code](https://mlsanity.com/paper/2006.06841/) (2020)
- [Machine Learning Security against Data Poisoning: Are We There Yet?](https://mlsanity.com/paper/2204.05986/) (2022)
- [Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning](https://mlsanity.com/paper/2107.05243/) (2021)
- [Poison Attack and Defense on Deep Source Code Processing Models](https://mlsanity.com/paper/2210.17029/) (2022)
- [Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators](https://mlsanity.com/paper/2212.06008/) (2022)
- [A Targeted Attack on Black-Box Neural Machine Translation with Parallel Data Poisoning](https://mlsanity.com/paper/2011.00675/) (2020)

## Built on by (most cited first, 6 of 6)

- [RedCoder: Automated Multi-Turn Red Teaming for Code LLMs](https://mlsanity.com/paper/2507.22063/) (2025)
- [MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?](https://mlsanity.com/paper/2507.19598/) (2025)
- [Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilities](https://mlsanity.com/paper/2509.25894/) (2025)
- [Optimizing AI-Assisted Code Generation](https://mlsanity.com/paper/2412.10953/) (2024)
- [Detecting Stealthy Data Poisoning Attacks in AI Code Generators](https://mlsanity.com/paper/2508.21636/) (2025)
- [Comprendia: AI-Augmented Code Comprehension](https://mlsanity.com/paper/2608.10290/) (2026)

## Abstract (arXiv)

AI-based code generators have gained a fundamental role in assisting developers in writing software starting from natural language (NL). However, since these large language models are trained on massive volumes of data collected from unreliable online sources (e.g., GitHub, Hugging Face), AI models become an easy target for data poisoning attacks, in which an attacker corrupts the training data by injecting a small amount of poison into it, i.e., astutely crafted malicious samples. In this position paper, we address the security of AI code generators by identifying a novel data poisoning attack that results in the generation of vulnerable code. Next, we devise an extensive evaluation of how these attacks impact state-of-the-art models for code generation. Lastly, we discuss potential solutions to overcome this threat.