mlsanityData to 14 Oct 2026

Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers

arXiv 2405.17527 ↗First posted 27 May 2024cs.LG cs.AI cs.NAcited by 35

Not on the map. Why?

Abstract (arXiv)

Deep models have recently emerged as promising tools to solve partial differential equations (PDEs), known as neural PDE solvers. While neural solvers trained from either simulation data or physics-informed loss can solve PDEs reasonably well, they are mainly restricted to a few instances of PDEs, e.g. a certain equation with a limited set of coefficients. This limits their generalization to diverse PDEs, preventing them from being practical surrogate models of numerical solvers. In this paper, we present Unisolver, a novel Transformer model trained on diverse data and conditioned on diverse PDEs, aiming towards a universal neural PDE solver capable of solving a wide scope of PDEs. Instead of purely scaling up data and parameters, Unisolver stems from the theoretical analysis of the PDE-solving process. Inspired by the mathematical structure of PDEs that a PDE solution is fundamentally governed by a series of PDE components such as equation symbols and boundary conditions, we define a complete set of PDE components and flexibly embed them as domain-wise and point-wise deep conditions for Transformer PDE solvers. Integrating physical insights with recent Transformer advances, Unisolver achieves consistent state-of-the-art on three challenging large-scale benchmarks, showing impressive performance and generalizability. Code is available at https://github.com/thuml/Unisolver.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

29 of 45 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  10. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  11. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
17 more
  1. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  2. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 527
  3. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 427
  4. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
  5. 2022Message Passing Neural PDE Solverscited by 386
  6. 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 326
  7. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 297
  8. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 197
  9. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 151
  10. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 131
  11. 2023Scalable Transformer for PDE Surrogate Modelingcited by 128
  12. 2023In-Context Operator Learning with Data Prompts for Differential Equation Problemscited by 101
  13. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 98
  14. 2023Learning Neural PDE Solvers with Parameter-Guided Channel Attentioncited by 38
  15. 2023Physics Informed Token Transformer for Solving Partial Differential Equationscited by 21
  16. 2023Polynomial Implicit Neural Representations For Large Diverse Datasetscited by 18
  17. 2023PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformerscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 25.

35 papers.

Sort:

By area:

  1. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 28
  2. 2025Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraintscited by 28
  3. 2026Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometriescited by 20
  4. 2025PhySense: Sensor Placement Optimization for Accurate Physics Sensingcited by 11
  5. 2026GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Trainingcited by 10
  6. 2025Text-Trained LLMs Can Zero-Shot Extrapolate PDE Dynamics, Revealing a Three-Stage In-Context Learning Mechanismcited by 7
  7. 2024Latent Neural Operator Pretraining for Solving Time-Dependent PDEscited by 5
  8. 2024MATEY: multiscale adaptive foundation models for spatiotemporal physical systemscited by 5
  9. 2026OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inferencecited by 5
  10. 2025SuperWing: a comprehensive transonic wing dataset for data-driven aerodynamic designcited by 3
  11. 2026Latent Generative Solvers for Generalizable Long-Term Physics Simulationcited by 3
  12. 2026NESTOR: A Nested MOE-based Neural Operator for Large-Scale PDE Pre-Trainingcited by 3
23 more
  1. 2025Efficient Generative Transformer Operators For Million-Point PDEscited by 2
  2. 2025Convolutional-neural-operator-based transfer learning for solving PDEscited by 2
  3. 2026MVNN: A Measure-Valued Neural Network for Learning McKean-Vlasov Dynamics from Particle Datacited by 1
  4. 2026Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospectscited by 1
  5. 2026Neural operator discovery from heterogeneous trajectoriescited by 1
  6. 2025Decoding Partial Differential Equations: Cross-Modal Adaptation of Decoder-only Models to PDEscited by 0
  7. 2026Brep2Shape: Boundary and Shape Representation Alignment via Self-Supervised Transformerscited by 0
  8. 2026Towards a Foundation-Model Paradigm for Aerodynamic Prediction in Three-dimensional Designcited by 0
  9. 2026Posterior-First Neural PDE Simulation: Inferring Hidden Problem State from a Single Fieldcited by 0
  10. 2026Replay-Based Continual Learning for Physics-Informed Neural Operatorscited by 0
  11. 2026CATO: Charted Attention for Neural PDE Operatorscited by 0
  12. 2026Large language models for partial differential equation workflowscited by 0
  13. 2026MoNo: Multiscale Optimal Transport Neural Operator for Solving PDEs on General Geometriescited by 0
  14. 2026Distillation of Foundation Models for Time-dependent PDEscited by 0
  15. 2026STCO: Conditional Neural Operators for Time-Dependent PDEscited by 0
  16. 2026Learnable composition for neural operatorscited by 0
  17. 2026Enhancing Transformer Representations of Symbolic ODE Expressionscited by 0
  18. 2026PhysFieldBench: Can Multimodal Models Understand Physical Fields?cited by 0
  19. 2026PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEscited by 0
  20. 2026Transolver-σ: Joint Spectral-Physical Subspace Modeling for Neural PDE Solvingcited by 0
  21. 2026Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIPcited by 0
  22. 2026SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretrainingcited by 0
  23. 2026PDE-PFN: Prior-Data Fitted Neural PDE Solvercited by 0