mlsanityData to 14 Oct 2026

PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generation

arXiv 2605.09636 ↗First posted 10 May 2026cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

PDE-to-solver code generation aims to automatically synthesize executable numerical solvers from partial differential equation (PDE) specifications. This task requires not only understanding the mathematical structure of PDEs, but also selecting appropriate discretization schemes and solver configurations, and correctly implementing the resulting formulations in finite-element method (FEM) libraries. Existing code generation benchmarks mainly evaluate syntactic correctness, or success on predefined test cases. To our knowledge, there is currently no publicly available benchmark specifically for PDE-to-solver code generation, and general-purpose code benchmarks do not fully capture the unique challenges of numerical PDE solution, such as ensuring solver accuracy, efficiency, and compatibility with professional FEM libraries. We introduce PDEAgent-Bench, to the best of our knowledge, the first multi-metric, multi-library benchmark for PDE-to-solver code generation. PDEAgent-Bench contains 645 instances across 6 mathematical categories and 11 PDE families, with common FEM libraries for DOLFINx, Firedrake, and deal.II. Each instance provides an agent-facing problem specification, a reference solution on a prescribed evaluation grid, and case-specific accuracy and runtime targets. PDEAgent-Bench adopts a staged evaluation framework in which generated solvers must sequentially pass executability, numerical accuracy, and computational efficiency checks. Experiments with representative LLMs and code agents show that models can often produce runnable code, but their pass rate drops substantially once accuracy and efficiency requirements are enforced. These results indicate that current agents remain limited in producing numerically reliable and efficient PDE solvers, and that PDEAgent-Bench provides a reproducible testbed grounded in the practical requirements of numerical PDE solving.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

35 of 46 references.

References · 35

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2021Program Synthesis with Large Language Modelscited by 3,637
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  9. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  10. 2021Measuring Coding Challenge Competence With APPScited by 902
  11. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  12. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
23 more
  1. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  2. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 197
  3. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
  4. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 109
  5. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 98
  6. 2023PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEscited by 87
  7. 2023ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Codecited by 65
  8. 2025MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Researchcited by 64
  9. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 61
  10. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 58
  11. 2023CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamicscited by 46
  12. 2025CodePDE: An Inference Framework for LLM-driven PDE Solver Generationcited by 39
  13. 2025ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Codecited by 36
  14. 2025GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveragingcited by 25
  15. 2025PDE-Controller: LLMs for Autoformalization and Reasoning of PDEscited by 23
  16. 2025PINNsAgent: Automated PDE Surrogation with Large Language Modelscited by 17
  17. 2026RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Datacited by 16
  18. 2024Explain Like I'm Five: Using LLMs to Improve PDE Surrogate Models with Textcited by 7
  19. 2025Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanismcited by 7
  20. 2025Self-Attention to Operator Learning-based 3D-IC Thermal Simulationcited by 7
  21. 2025Automated Code Development for PDE Solvers Using Large Language Modelscited by 6
  22. 2026OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inferencecited by 5
  23. 2026RACE-Bench: A Reasoning-Augmented Benchmark for Repository-Level Code Agents on Feature Additioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Large language models for partial differential equation workflowscited by 0
  2. 2026AutoPDEBench: Benchmarking LLM Auto-Research for Neural PDE Solver Designcited by 0
  3. 2026GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulationcited by 0
  4. 2026Rules to Tools: Executable Checks for LLM Agents in Scientific Computingcited by 0
  5. 2026Training Numerical Intelligence via Auto-Diagnosis and Skill Discoverycited by 0
  6. 2026Language models can notice an impossible engineering problem yet still report it as solvedcited by 0
  7. 2026WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?cited by 0
  8. 2026Transferability of Learned States in Neural PDE Solverscited by 0