mlsanityData to 7 Oct 2026

CodePDE: An Inference Framework for LLM-driven PDE Solver Generation

arXiv 2505.08783 ↗First posted 13 May 2025cs.LG cs.AI cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Partial differential equations (PDEs) are fundamental to modeling physical systems, yet solving them remains a complex challenge. Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive, while neural-network-based solvers require large training datasets and often lack interpretability. In this work, we frame PDE solving as a code generation task and introduce CodePDE, the first inference framework for generating PDE solvers using large language models (LLMs). With CodePDE, we present a thorough evaluation on critical capacities of LLM for PDE solving: reasoning, debugging, self-refinement, and test-time scaling. CodePDE shows that, with advanced inference-time algorithms and scaling strategies, LLMs can achieve strong performance across a range of representative PDE problems. We also identify novel insights into LLM-driven solver generation, such as trade-offs between solver reliability and sophistication, design principles for LLM-powered PDE solving agents, and failure modes for LLM on hard tasks. These insights offer guidance for building more capable and reliable LLM-based scientific engines.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 47 of its 79 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

47 of 79 references.

References · 47

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  4. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  5. 2023Teaching Large Language Models to Self-Debugcited by 963
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  8. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  9. 2021Machine learning accelerated computational fluid dynamicscited by 461
  10. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  11. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  12. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
35 more
  1. 2022Message Passing Neural PDE Solverscited by 368
  2. 2021Choose a Transformer: Fourier or Galerkincited by 333
  3. 2017PDE-Net: Learning PDEs from Datacited by 304
  4. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
  5. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  6. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
  7. 2024Poseidon: Efficient Foundation Models for PDEscited by 186
  8. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 179
  9. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 172
  10. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  11. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
  12. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 136
  13. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
  14. 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 82
  15. 2024ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Datacited by 39
  16. 2023Cross-Modal Fine-Tuning: Align then Refinecited by 34
  17. 2021NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Taskscited by 32
  18. 2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domainscited by 30
  19. 2024On the Benefits of Memory for Modeling Time-Dependent PDEscited by 30
  20. 2025CO-Bench: Benchmarking Language Model Agents in Algorithm Search for Combinatorial Optimizationcited by 28
  21. 2024UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptationcited by 27
  22. 2023BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Modelscited by 24
  23. 2022Learning Physics-Informed Neural Networks without Stacked Back-propagationcited by 23
  24. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 23
  25. 2025PDE-Controller: LLMs for Autoformalization and Reasoning of PDEscited by 23
  26. 2022Guiding continuous operator learning through Physics-based boundary constraintscited by 21
  27. 2024Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventionscited by 20
  28. 2023Deep Equilibrium Based Neural Operators for Steady-State PDEscited by 19
  29. 2022Efficient Architecture Search for Diverse Taskscited by 18
  30. 2025CAT: Content-Adaptive Image Tokenizationcited by 18
  31. 2020Emergence of Pragmatics from Referential Game between Theory of Mind Agentscited by 17
  32. 2024Meta-Designing Quantum Experiments with Language Modelscited by 12
  33. 2021Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximationcited by 10
  34. 2025Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsitycited by 7
  35. 2021Iterative Teacher-Aware Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 23.

37 papers.

Sort:

By area:

  1. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  2. 2026DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolutioncited by 23
  3. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 22
  4. 2025Scientific Algorithm Discovery by Augmenting AlphaEvolve with Deep Researchcited by 16
  5. 2025Sample Complexity and Representation Ability of Test-time Scaling Paradigmscited by 12
  6. 2025Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Frameworkcited by 11
  7. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  8. 2025Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanismcited by 7
  9. 2026AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computingcited by 7
  10. 2026GRAFT-ATHENA: Self-Improving Agentic Teams for Autonomous Discovery and Evolutionary Numerical Algorithmscited by 6
  11. 2025CoMind: Towards Community-Driven Agents for Machine Learning Engineeringcited by 5
  12. 2025SciML Agents: Write the Solver, Not the Solutioncited by 5
25 more
  1. 2026ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Responsecited by 5
  2. 2026OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inferencecited by 4
  3. 2026Agentic Scientific Simulation: Execution-Grounded Model Construction and Reconstructioncited by 4
  4. 2026PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generationcited by 4
  5. 2025PDE-SHARP: PDE Solver Hybrids through Analysis and Refinement Passescited by 3
  6. 2026SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMscited by 3
  7. 2025Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revisioncited by 2
  8. 2025RECODE: Reasoning Through Code Generation for Visual Question Answeringcited by 2
  9. 2025Automating modeling in mechanics: LLMs as designers of physics-constrained neural networks for constitutive modeling of materialscited by 2
  10. 2026AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategiescited by 2
  11. 2026ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issuescited by 2
  12. 2025Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operatorscited by 1
  13. 2026EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networkscited by 1
  14. 2025Decoding Partial Differential Equations: Cross-Modal Adaptation of Decoder-only Models to PDEscited by 0
  15. 2026Protein Design with Agent Rosetta: A Case Study for Specialized Scientific Agentscited by 0
  16. 2026LLM-driven design of physics-constrained constitutive models: two agents are better than onecited by 0
  17. 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branchescited by 0
  18. 2026Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewardscited by 0
  19. 2026Large language models for partial differential equation workflowscited by 0
  20. 2026Improving Auto-Design of Neural PDE Solvers with a Domain-Specific Languagecited by 0
  21. 2026DiffPDE: Masked Diffusion Language Models as PDE Solvercited by 0
  22. 2026GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulationcited by 0
  23. 2026Rules to Tools: Executable Checks for LLM Agents in Scientific Computingcited by 0
  24. 2026LLM-Assisted Generation of Transparent, Open-Source Multiphysics Models of Electrochemical Devicescited by 0
  25. 2026Prior or Feedback? What an LLM Uses When Adapting Neural Operatorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Automated Code Development for PDE Solvers Using Large Language Models
  2. 2025Towards Reasoning for PDE Foundation Models: A Reward-Model-Driven Inference-Time-Scaling Algorithm
  3. 2026AutoPDEBench: Benchmarking LLM Auto-Research for Neural PDE Solver Design
  4. 2025PDE-Agent: A toolchain-augmented multi-agent framework for PDE solving
  5. 2024Text2PDE: Latent Diffusion Models for Accessible Physics Simulation
  6. 2026PDEFlow: Autonomous Agentic PDE Pipelines for Neural Operator Learning and Solver-Free Inference
  7. 2025Physics-Informed Inference Time Scaling for Solving High-Dimensional PDE via Defect Correction
  8. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers