mlsanityData to 14 Oct 2026

The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

arXiv 2609.33078 ↗First posted 27 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Automatic differentiation (AD) lets neural networks compute derivatives of governing equations to machine precision, and this precision has made it the computational backbone of physics-informed machine learning. Yet exactness in the mathematical sense is not the same as fidelity to the physics. Here I argue that a derivative can be numerically perfect and still be the wrong derivative for the problem at hand, because AD, by construction, has no notion of the physical structure a solution must obey. Convection and its associated directionality, diffusion, and dispersion are only the most visible instances of a much longer list that spans all branches of computational science and engineering, including conservation, thermodynamic consistency, symmetry, symplectic structure, positivity, monotonicity, and boundedness. Recognizing this broader gap reframes how the field should build the next generation of PDE-driven neural surrogates.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 83 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

35 of 83 references.

References · 35

  1. 2016Group Equivariant Convolutional Networkscited by 1,165
  2. 2015Automatic differentiation in machine learning: a surveycited by 855
  3. 2019Hamiltonian Neural Networkscited by 712
  4. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  5. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  6. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  7. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  8. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 540
  9. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 527
  10. 2020Lagrangian Neural Networkscited by 398
  11. 2021Physics-informed neural networks with hard constraints for inverse designcited by 280
  12. 2019Symplectic Recurrent Neural Networkscited by 150
23 more
  1. 2021Exact imposition of boundary conditions with distance functions in physics-informed deep neural networkscited by 137
  2. 2018Physics-Informed Generative Adversarial Networks for Stochastic Differential Equationscited by 135
  3. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  4. 2024A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networkscited by 104
  5. 2021CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Methodcited by 93
  6. 2022Physics-informed neural networks for inverse problems in supersonic flowscited by 76
  7. 2022Learning two-phase microstructure evolution using neural operators and autoencoder architecturescited by 72
  8. 2023A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositionscited by 48
  9. 2023Physics-Informed Computer Vision: A Review and Perspectivescited by 28
  10. 2022Accelerated Training of Physics-Informed Neural Networks (PINNs) using Meshless Discretizationscited by 27
  11. 2023Lie Point Symmetry and Physics Informed Networkscited by 24
  12. 2024Accelerating Phase Field Simulations Through a Hybrid Adaptive Fourier Neural Operator with U-Net Backbonecited by 10
  13. 2023Improving physics-informed DeepONets with hard constraintscited by 9
  14. 2024Energy Dissipation Preserving Physics Informed Neural Network for Allen-Cahn Equationscited by 6
  15. 2026FEKAN: Feature-Enriched Kolmogorov-Arnold Networkscited by 5
  16. 2024An invariance constrained deep learning network for PDE discoverycited by 3
  17. 2024WANCO: Weak Adversarial Networks for Constrained Optimization problemscited by 3
  18. 2026Symplectic Neural Operators for Learning Infinite Dimensional Hamiltonian Systemscited by 3
  19. 2025Neural non-canonical Hamiltonian dynamics for long-time simulationscited by 2
  20. 2026Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradientscited by 2
  21. 2024Symmetry group based domain decomposition to enhance physics-informed neural networks for solving partial differential equationscited by 1
  22. 2024Deep Neural Networks with Symplectic Preservation Propertiescited by 1
  23. 2026GenVoid: Uncertainty-Aware Learning of Subsurface Material Defects with an Experimentally Validated Physics-Informed Generative Modelcited by 1

Built on by

Not cited yet.