# Curvature-aware dynamic precision approach for physics-informed neural networks

arXiv: https://arxiv.org/abs/2606.04736 · HTML: https://mlsanity.com/paper/2606.04736/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 69 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 30 of 36)

- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [An Investigation into Neural Net Optimization via Hessian Eigenvalue Density](https://mlsanity.com/paper/1901.10159/) (2019)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [MLPerf Training Benchmark](https://mlsanity.com/paper/1910.01500/) (2019)
- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [Efficient training of physics-informed neural networks via importance sampling](https://mlsanity.com/paper/2104.12325/) (2021)
- [PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2306.08827/) (2023)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks](https://mlsanity.com/paper/2406.11045/) (2024)
- [DAS-PINNs: A deep adaptive sampling method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2112.14038/) (2021)
- [Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective](https://mlsanity.com/paper/2502.00604/) (2025)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?](https://mlsanity.com/paper/2405.04230/) (2024)
- [Physics-informed neural networks for solving forward and inverse problems in complex beam systems](https://mlsanity.com/paper/2303.01055/) (2023)
- [Mixed Precision Training With 8-bit Floating Point](https://mlsanity.com/paper/1905.12334/) (2019)
- [Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark](https://mlsanity.com/paper/1806.01427/) (2018)
- [An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions](https://mlsanity.com/paper/2309.07049/) (2023)
- [Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks](https://mlsanity.com/paper/2407.01613/) (2024)
- [Multi-Precision Policy Enforced Training (MuPPET): A precision-switching strategy for quantised fixed-point training of CNNs](https://mlsanity.com/paper/2006.09049/) (2020)
- [Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks](https://mlsanity.com/paper/2604.05230/) (2026)
- [PAD-Net: An Efficient Framework for Dynamic Networks](https://mlsanity.com/paper/2211.05528/) (2022)
- [Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs](https://mlsanity.com/paper/2107.13490/) (2021)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have become a promising framework for simulating partial differential equations (PDEs) by embedding physical laws directly into neural network training. However, recent studies show that PINN optimisation is sensitive to numerical precision. Existing implementations commonly use either single precision (FP32), which is computationally efficient but prone to failure modes, or double precision (FP64), which is robust but substantially expensive. This creates a trade-off between computational efficiency and numerical accuracy. To reduce the computational cost of double-precision training while retaining prediction accuracy, we propose a curvature-aware precision controller that adapts numerical precision during training rather than treating it as a fixed implementation choice. The proposed method reuses curvature information derived from the limited-memory BFGS (L-BFGS) optimiser to construct a precision controller, retaining FP32 when lower precision is sufficient and promoting computation to FP64 when the training dynamics indicate numerical sensitivity or precision-limited stagnation. We evaluate the proposed approach on four canonical PINN failure-mode benchmarks and an irradiance-driven ordinary differential equation example. We further test the proposed approach across different neural network architectures. The method consistently matches or even slightly exceeds full FP64 solution accuracy while reducing training time relative to full double-precision training on all benchmark equations. The obtained results indicate that precision sensitivity in PINN optimisation is phase-dependent, and that selectively applying higher precision only during numerically critical stages can lower computational cost without sacrificing predictive accuracy.