# EVIL: Exploiting Software via Natural Language

arXiv: https://arxiv.org/abs/2109.00279 · HTML: https://mlsanity.com/paper/2109.00279/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2021.

## Built on by (most cited first, 12 of 12)

- [Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks](https://mlsanity.com/paper/2308.04451/) (2023)
- [Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators](https://mlsanity.com/paper/2212.06008/) (2022)
- [Automating the Correctness Assessment of AI-generated Code for Security Contexts](https://mlsanity.com/paper/2310.18834/) (2023)
- [Defending Code Language Models against Backdoor Attacks with Deceptive Cross-Entropy Loss](https://mlsanity.com/paper/2407.08956/) (2024)
- [Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Quality](https://mlsanity.com/paper/2503.05860/) (2025)
- [A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspective](https://mlsanity.com/paper/2202.04847/) (2022)
- [Enhancing Robustness of AI Offensive Code Generators via Data Augmentation](https://mlsanity.com/paper/2306.05079/) (2023)
- [Enhancing AI-based Generation of Software Exploits with Contextual Information](https://mlsanity.com/paper/2408.02402/) (2024)
- [Can NMT Understand Me? Towards Perturbation-based Evaluation of NMT Models for Code Generation](https://mlsanity.com/paper/2203.15319/) (2022)
- [Optimizing AI-Assisted Code Generation](https://mlsanity.com/paper/2412.10953/) (2024)
- [Can We Generate Shellcodes via Natural Language? An Empirical Study](https://mlsanity.com/paper/2202.03755/) (2022)
- [Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration](https://mlsanity.com/paper/2606.05396/) (2026)

## Abstract (arXiv)

Writing exploits for security assessment is a challenging task. The writer needs to master programming and obfuscation techniques to develop a successful exploit. To make the task easier, we propose an approach (EVIL) to automatically generate exploits in assembly/Python language from descriptions in natural language. The approach leverages Neural Machine Translation (NMT) techniques and a dataset that we developed for this work. We present an extensive experimental study to evaluate the feasibility of EVIL, using both automatic and manual analysis, and both at generating individual statements and entire exploits. The generated code achieved high accuracy in terms of syntactic and semantic correctness.