mlsanityData to 14 Oct 2026

Your Simulation Runs but Solves the Wrong Physics: PDE-Grounded Intent Verification for LLM-Generated Multiphysics Simulation Code

arXiv 2605.09360 ↗First posted 10 May 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Execution-based evaluation of LLM-generated code implicitly treats successful execution as a proxy for correctness. In scientific simulation, this proxy is insufficient: a generated input file can run, mesh, and converge while encoding governing equations that differ from the user's intent. We call this mismatch between intended physics and generated code the comprehension-generation gap. We instantiate this in MOOSE, where Kernel and BC objects map compositionally to weak-form residual terms, enabling deterministic reconstruction of the encoded PDE and comparison against an intended contract. We formalize this comparison as the Intent Fidelity Score (IFS), a structural metric covering governing terms, BCs, ICs, coefficients, and time scheme. Building on IFS, we develop a PDE-grounded refinement loop that uses deterministic violation reports to correct generated code iteratively. We evaluate on MooseBench, a 220-case multiphysics benchmark with PDE-level ground truth released with this work. On this benchmark, our method consistently improves mean IFS over direct generation, with gains concentrated on hard cases. On the subset where direct generation falls below IFS 0.7, refinement adds +0.22 to +0.41 absolute IFS. In the deployment audit, execution-only repair improves execution success while leaving 39-40% of all 220 cases runnable but still solving the wrong physics across the three main deployment-audit models, exposing executability and intent fidelity as separable failure modes. Static proof-of-concept experiments on four PDE-oriented DSLs (UFL/FEniCS, FreeFEM, FiPy, and Devito) suggest that the reconstruction-and-comparison pattern extends beyond MOOSE. These findings reinforce that executable simulation code should be verified against the mathematical structure it is intended to encode, not accepted on execution alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 25 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

11 of 25 references.

References · 11

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  3. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  4. 2023Teaching Large Language Models to Self-Debugcited by 988
  5. 2019Deep Learning for Symbolic Mathematicscited by 270
  6. 2023MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledgecited by 63
  7. 2024MetaOpenFOAM: an LLM-based multi-agent framework for CFDcited by 39
  8. 2026TerraFormer: Automated Infrastructure-as-Code with LLMs Fine-Tuned via Policy-Guided Verifier Feedbackcited by 5
  9. 2024AutoFLUKA: A Large Language Model Based Framework for Automating Monte Carlo Simulations in FLUKAcited by 3
  10. 2026MOOSEnger: A Simulation-Aware AI Agent Framework for the MOOSE Ecosystemcited by 2
  11. 2025Deployability-Centric Infrastructure-as-Code Generation: Fail, Learn, Refine, and Succeed through LLM-Empowered DevOps Simulationcited by 1

Built on by

Not cited yet.