mlsanityData to 14 Oct 2026

A semantic mutation metric for metamorphic relation adequacy in scientific computing programs

arXiv 2605.17437 ↗First posted 17 May 2026cs.SE cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Context. Metamorphic Testing addresses the test-oracle problem in scientific computing, but classical Mutation Score operates on syntactic AST mutations and misses domain semantics. Objective. We propose the Semantic Mutation Score (SMS), built on five domain-semantic operators (Conservation Erosion, Operator Substitution, Hyperparameter, Trajectory Flip, Structural Injection). SMS degenerates almost everywhere to MS in a characterised limit, so any SMS-based conclusion remains consistent with prior mutation-testing literature in the classical regime. Method. A 12-PUT x 5-MP design over four single-output float-to-float classes (numeric, probabilistic, surrogate, machine-learning) is paired with a three-layer attribution classifier separating true semantic faults from tolerance, OOD, statistical, and artefact categories. A same-source / cross-source ablation under an identical prompt isolates the LLM-source-diversity contribution. LLM-generated mutants are compared against a default-configuration cosmic-ray syntactic pool at the AST-normalised level. Results. The pre-registered large-effect threshold for Cliff's delta is not met under the point-estimate criterion; the observed effect lies in the medium-effect range. Cross-source pooling under an identical prompt does not appreciably shift delta, indicating that LLM identity is not the lever within this design. AST-level overlap between LLM-generated and default cosmic-ray syntactic mutants is small; the Hyperparameter, Structural Injection, and Trajectory Flip classes are unreachable under default first-order syntactic configurations. Conclusion. SMS is a backward-compatible adequacy metric for domain-semantic metamorphic-relation sets in scientific computing. The first-order unreachability evidence is independent of the effect-size question.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 47 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

4 of 47 references.

References · 4

  1. 2023Effective Test Generation Using Pre-trained Large Language Models and Mutation Testingcited by 51
  2. 2024LLMorpheus: Mutation Testing using Large Language Modelscited by 8
  3. 2021Does mutation testing improve testing practices?cited by 2
  4. 2013A Semantics for Approximate Program Transformationscited by 1

Built on by

Not cited yet.