# Towards Generating Executable Metamorphic Relations Using Large Language Models

arXiv: https://arxiv.org/abs/2401.17019 · HTML: https://mlsanity.com/paper/2401.17019/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2024.

## Built on by (most cited first, 7 of 7)

- [Metamorphic Testing of Large Language Models for Natural Language Processing](https://mlsanity.com/paper/2511.02108/) (2025)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [AutoMT: A Multi-Agent LLM Framework for Automated Metamorphic Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2510.19438/) (2025)
- [MR-Coupler: Automated Metamorphic Test Generation via Functional Coupling Analysis](https://mlsanity.com/paper/2604.10126/) (2026)
- [Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations](https://mlsanity.com/paper/2605.25101/) (2026)
- [Multi-Agent LLM-based Metamorphic Testing for REST APIs](https://mlsanity.com/paper/2605.28321/) (2026)
- [Multi-Large Language Model Orchestrated Severity Assessment of Clinical Records (MOSAIC)](https://mlsanity.com/paper/2607.05032/) (2026)

## Abstract (arXiv)

Metamorphic testing (MT) has proven to be a successful solution to automating testing and addressing the oracle problem. However, it entails manually deriving metamorphic relations (MRs) and converting them into an executable form; these steps are time-consuming and may prevent the adoption of MT. In this paper, we propose an approach for automatically deriving executable MRs (EMRs) from requirements using large language models (LLMs). Instead of merely asking the LLM to produce EMRs, our approach relies on a few-shot prompting strategy to instruct the LLM to perform activities in the MT process, by providing requirements and API specifications, as one would do with software engineers. To assess the feasibility of our approach, we conducted a questionnaire-based survey in collaboration with Siemens Industry Software, a worldwide leader in providing industry software and services, focusing on four of their software applications. Additionally, we evaluated the accuracy of the generated EMRs for a Web application. The outcomes of our study are highly promising, as they demonstrate the capability of our approach to generate MRs and EMRs that are both comprehensible and pertinent for testing purposes.