# Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators

arXiv: https://arxiv.org/abs/2212.06008 · HTML: https://mlsanity.com/paper/2212.06008/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 85 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [IntelliCode Compose: Code Generation Using Transformer](https://mlsanity.com/paper/2005.08025/) (2020)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow](https://mlsanity.com/paper/1805.08949/) (2018)
- [PyMT5: multi-mode translation of natural language and Python code with transformers](https://mlsanity.com/paper/2010.03150/) (2020)
- [Latent Predictor Networks for Code Generation](https://mlsanity.com/paper/1603.06744/) (2016)
- [Out of the BLEU: how should we assess quality of the Code Generation models?](https://mlsanity.com/paper/2208.03133/) (2022)
- [CoTexT: Multi-task Learning with Code-Text Transformer](https://mlsanity.com/paper/2105.08645/) (2021)
- [NatGen: Generative pre-training by "Naturalizing" source code](https://mlsanity.com/paper/2206.07585/) (2022)
- [Does BLEU Score Work for Code Migration?](https://mlsanity.com/paper/1906.04903/) (2019)
- [JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation](https://mlsanity.com/paper/1910.02216/) (2019)
- [Applying CodeBERT for Automated Program Repair of Java Simple Bugs](https://mlsanity.com/paper/2103.11626/) (2021)
- [Multilingual training for Software Engineering](https://mlsanity.com/paper/2112.02043/) (2021)
- [No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence](https://mlsanity.com/paper/2207.11680/) (2022)
- [The Threat of Offensive AI to Organizations](https://mlsanity.com/paper/2106.15764/) (2021)
- [Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation](https://mlsanity.com/paper/2204.00447/) (2022)
- [Translation Quality Assessment: A Brief Survey on Manual and Automatic Methods](https://mlsanity.com/paper/2105.03311/) (2021)
- [Shellcode_IA32: A Dataset for Automatic Shellcode Generation](https://mlsanity.com/paper/2104.13100/) (2021)
- [XNMT: The eXtensible Neural Machine Translation Toolkit](https://mlsanity.com/paper/1803.00188/) (2018)

## Built on by (most cited first, 11 of 11)

- [Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks](https://mlsanity.com/paper/2308.04451/) (2023)
- [Assessing Evaluation Metrics for Neural Test Oracle Generation](https://mlsanity.com/paper/2310.07856/) (2023)
- [Automating the Correctness Assessment of AI-generated Code for Security Contexts](https://mlsanity.com/paper/2310.18834/) (2023)
- [Poisoning Programs by Un-Repairing Code: Security Concerns of AI-generated Code](https://mlsanity.com/paper/2403.06675/) (2024)
- [Enhancing Robustness of AI Offensive Code Generators via Data Augmentation](https://mlsanity.com/paper/2306.05079/) (2023)
- [Enhancing AI-based Generation of Software Exploits with Contextual Information](https://mlsanity.com/paper/2408.02402/) (2024)
- [AI Code Generators for Security: Friend or Foe?](https://mlsanity.com/paper/2402.01219/) (2024)
- [CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation](https://mlsanity.com/paper/2505.19502/) (2025)
- [The illusion of a perfect metric: Why evaluating AI's words is harder than it looks](https://mlsanity.com/paper/2508.13816/) (2025)
- [Detecting Stealthy Data Poisoning Attacks in AI Code Generators](https://mlsanity.com/paper/2508.21636/) (2025)
- [Can Neural Decompilation Assist Vulnerability Prediction on Binary Code?](https://mlsanity.com/paper/2412.07538/) (2024)

## Abstract (arXiv)

AI-based code generators are an emerging solution for automatically writing programs starting from descriptions in natural language, by using deep neural networks (Neural Machine Translation, NMT). In particular, code generators have been used for ethical hacking and offensive security testing by generating proof-of-concept attacks. Unfortunately, the evaluation of code generators still faces several issues. The current practice uses output similarity metrics, i.e., automatic metrics that compute the textual similarity of generated code with ground-truth references. However, it is not clear what metric to use, and which metric is most suitable for specific contexts. This work analyzes a large set of output similarity metrics on offensive code generators. We apply the metrics on two state-of-the-art NMT models using two datasets containing offensive assembly and Python code with their descriptions in the English language. We compare the estimates from the automatic metrics with human evaluation and provide practical insights into their strengths and limitations.