# MaPPing Your Model: Assessing the Impact of Adversarial Attacks on LLM-based Programming Assistants

arXiv: https://arxiv.org/abs/2407.11072 · HTML: https://mlsanity.com/paper/2407.11072/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 12 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Poisoning Language Models During Instruction Tuning](https://mlsanity.com/paper/2305.00944/) (2023)
- [Do Users Write More Insecure Code with AI Assistants?](https://mlsanity.com/paper/2211.03622/) (2022)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [Towards A Unified Agent with Foundation Models](https://mlsanity.com/paper/2307.09668/) (2023)
- [DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions](https://mlsanity.com/paper/2312.04730/) (2023)
- [The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers](https://mlsanity.com/paper/2404.02806/) (2024)

## Built on by (most cited first, 5 of 5)

- [RedCoder: Automated Multi-Turn Red Teaming for Code LLMs](https://mlsanity.com/paper/2507.22063/) (2025)
- [Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilities](https://mlsanity.com/paper/2509.25894/) (2025)
- [SOK: Exploring Hallucinations and Security Risks in AI-Assisted Software Development with Insights for LLM Deployment](https://mlsanity.com/paper/2502.18468/) (2025)
- [A Survey: Towards Privacy and Security in Mobile Large Language Models](https://mlsanity.com/paper/2509.02411/) (2025)
- [IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests](https://mlsanity.com/paper/2607.20759/) (2026)

## Abstract (arXiv)

LLM-based programming assistants offer the promise of programming faster but with the risk of introducing more security vulnerabilities. Prior work has studied how LLMs could be maliciously fine-tuned to suggest vulnerabilities more often. With the rise of agentic LLMs, which may use results from an untrusted third party, there is a growing risk of attacks on the model's prompt. We introduce the Malicious Programming Prompt (MaPP) attack, in which an attacker adds a small amount of text to a prompt for a programming task (under 500 bytes). We show that our prompt strategy can cause an LLM to add vulnerabilities while continuing to write otherwise correct code. We evaluate three prompts on seven common LLMs, from basic to state-of-the-art commercial models. Using the HumanEval benchmark, we find that our prompts are broadly effective, with no customization required for different LLMs. Furthermore, the LLMs that are best at HumanEval are also best at following our malicious instructions, suggesting that simply scaling language models will not prevent MaPP attacks. Using a dataset of eight CWEs in 16 scenarios, we find that MaPP attacks are also effective at implementing specific and targeted vulnerabilities across a range of models. Our work highlights the need to secure LLM prompts against manipulation as well as rigorously auditing code generated with the help of LLMs.