# CodePDE: An Inference Framework for LLM-driven PDE Solver Generation

arXiv: https://arxiv.org/abs/2505.08783 · HTML: https://mlsanity.com/paper/2505.08783/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 79 references that are in the map.
- First posted on arXiv 13 May 2025.

## Builds on (most cited first, 30 of 47)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks](https://mlsanity.com/paper/2305.18365/) (2023)
- [SciCode: A Research Coding Benchmark Curated by Scientists](https://mlsanity.com/paper/2407.13168/) (2024)
- [MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2310.03731/) (2023)
- [DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training](https://mlsanity.com/paper/2403.03542/) (2024)
- [From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models](https://mlsanity.com/paper/2406.16838/) (2024)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery](https://mlsanity.com/paper/2405.09783/) (2024)
- [ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Data](https://mlsanity.com/paper/2411.15004/) (2024)
- [Cross-Modal Fine-Tuning: Align then Refine](https://mlsanity.com/paper/2302.05738/) (2023)
- [NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks](https://mlsanity.com/paper/2110.05668/) (2021)
- [On the Benefits of Memory for Modeling Time-Dependent PDEs](https://mlsanity.com/paper/2409.02313/) (2024)

## Built on by (most cited first, 30 of 39)

- [Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction](https://mlsanity.com/paper/2506.07976/) (2025)
- [DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution](https://mlsanity.com/paper/2602.02919/) (2026)
- [ReCreate: Reasoning and Creating Domain Agents Driven by Experience](https://mlsanity.com/paper/2601.11100/) (2026)
- [Scientific Algorithm Discovery by Augmenting AlphaEvolve with Deep Research](https://mlsanity.com/paper/2510.06056/) (2025)
- [Sample Complexity and Representation Ability of Test-time Scaling Paradigms](https://mlsanity.com/paper/2506.05295/) (2025)
- [Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Framework](https://mlsanity.com/paper/2510.05158/) (2025)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing](https://mlsanity.com/paper/2602.17607/) (2026)
- [PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generation](https://mlsanity.com/paper/2605.09636/) (2026)
- [Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanism](https://mlsanity.com/paper/2509.06322/) (2025)
- [SciML Agents: Write the Solver, Not the Solution](https://mlsanity.com/paper/2509.09936/) (2025)
- [ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response](https://mlsanity.com/paper/2604.21199/) (2026)
- [GRAFT-ATHENA: Self-Improving Agentic Teams for Autonomous Discovery and Evolutionary Numerical Algorithms](https://mlsanity.com/paper/2605.11117/) (2026)
- [CoMind: Towards Community-Driven Agents for Machine Learning Engineering](https://mlsanity.com/paper/2506.20640/) (2025)
- [OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inference](https://mlsanity.com/paper/2602.01493/) (2026)
- [Agentic Scientific Simulation: Execution-Grounded Model Construction and Reconstruction](https://mlsanity.com/paper/2603.00214/) (2026)
- [PDE-SHARP: PDE Solver Hybrids through Analysis and Refinement Passes](https://mlsanity.com/paper/2511.00183/) (2025)
- [SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMs](https://mlsanity.com/paper/2603.20253/) (2026)
- [AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategies](https://mlsanity.com/paper/2606.10752/) (2026)
- [Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revision](https://mlsanity.com/paper/2508.20729/) (2025)
- [RECODE: Reasoning Through Code Generation for Visual Question Answering](https://mlsanity.com/paper/2510.13756/) (2025)
- [Automating modeling in mechanics: LLMs as designers of physics-constrained neural networks for constitutive modeling of materials](https://mlsanity.com/paper/2512.01735/) (2025)
- [ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues](https://mlsanity.com/paper/2606.18237/) (2026)
- [Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards](https://mlsanity.com/paper/2607.10474/) (2026)
- [EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks](https://mlsanity.com/paper/2607.26490/) (2026)
- [Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators](https://mlsanity.com/paper/2506.19396/) (2025)
- [Decoding Partial Differential Equations: Cross-Modal Adaptation of Decoder-only Models to PDEs](https://mlsanity.com/paper/2510.05278/) (2025)
- [Protein Design with Agent Rosetta: A Case Study for Specialized Scientific Agents](https://mlsanity.com/paper/2603.15952/) (2026)
- [LLM-driven design of physics-constrained constitutive models: two agents are better than one](https://mlsanity.com/paper/2605.23754/) (2026)
- [Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches](https://mlsanity.com/paper/2606.01145/) (2026)

## Abstract (arXiv)

Partial differential equations (PDEs) are fundamental to modeling physical systems, yet solving them remains a complex challenge. Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive, while neural-network-based solvers require large training datasets and often lack interpretability. In this work, we frame PDE solving as a code generation task and introduce CodePDE, the first inference framework for generating PDE solvers using large language models (LLMs). With CodePDE, we present a thorough evaluation on critical capacities of LLM for PDE solving: reasoning, debugging, self-refinement, and test-time scaling. CodePDE shows that, with advanced inference-time algorithms and scaling strategies, LLMs can achieve strong performance across a range of representative PDE problems. We also identify novel insights into LLM-driven solver generation, such as trade-offs between solver reliability and sophistication, design principles for LLM-powered PDE solving agents, and failure modes for LLM on hard tasks. These insights offer guidance for building more capable and reliable LLM-based scientific engines.