# Validating LLM-Generated Programs with Metamorphic Prompt Testing

arXiv: https://arxiv.org/abs/2406.06864 · HTML: https://mlsanity.com/paper/2406.06864/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [OctoPack: Instruction Tuning Code Large Language Models](https://mlsanity.com/paper/2308.07124/) (2023)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation](https://mlsanity.com/paper/2212.01020/) (2022)
- [Paraphrasing with Large Language Models](https://mlsanity.com/paper/1911.09661/) (2019)
- [Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches](https://mlsanity.com/paper/2009.07740/) (2020)
- [Unveiling Hidden DNN Defects with Decision-Based Metamorphic Testing](https://mlsanity.com/paper/2210.04942/) (2022)

## Built on by (most cited first, 6 of 6)

- [TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models](https://mlsanity.com/paper/2407.21227/) (2024)
- [Bridging LLM-Generated Code and Requirements: Reverse Generation technique and SBC Metric for Developer Insights](https://mlsanity.com/paper/2502.07835/) (2025)
- [VeriSimpl: Robust Optimization Modeling from Natural Language using Simplification-based Verification](https://mlsanity.com/paper/2607.20474/) (2026)
- [Multi-Agent Code Verification via Information Theory](https://mlsanity.com/paper/2511.16708/) (2025)
- [Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits](https://mlsanity.com/paper/2607.16646/) (2026)
- [SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Histories](https://mlsanity.com/paper/2608.09799/) (2026)

## Abstract (arXiv)

The latest paradigm shift in software development brings in the innovation and automation afforded by Large Language Models (LLMs), showcased by Generative Pre-trained Transformer (GPT), which has shown remarkable capacity to generate code autonomously, significantly reducing the manual effort required for various programming tasks. Although, the potential benefits of LLM-generated code are vast, most notably in efficiency and rapid prototyping, as LLMs become increasingly integrated into the software development lifecycle and hence the supply chain, complex and multifaceted challenges arise as the code generated from these language models carry profound questions on quality and correctness. Research is required to comprehensively explore these critical concerns surrounding LLM-generated code. In this paper, we propose a novel solution called metamorphic prompt testing to address these challenges. Our intuitive observation is that intrinsic consistency always exists among correct code pieces but may not exist among flawed code pieces, so we can detect flaws in the code by detecting inconsistencies. Therefore, we can vary a given prompt to multiple prompts with paraphrasing, and to ask the LLM to acquire multiple versions of generated code, so that we can validate whether the semantic relations still hold in the acquired code through cross-validation. Our evaluation on HumanEval shows that metamorphic prompt testing is able to detect 75 percent of the erroneous programs generated by GPT-4, with a false positive rate of 8.6 percent.