# From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNs

arXiv: https://arxiv.org/abs/2608.04206 · HTML: https://mlsanity.com/paper/2608.04206/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks](https://mlsanity.com/paper/1905.01205/) (2019)
- [Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective](https://mlsanity.com/paper/2502.00604/) (2025)
- [Solving Inverse Stochastic Problems from Discrete Particle Observations Using the Fokker-Planck Equation and Physics-informed Neural Networks](https://mlsanity.com/paper/2008.10653/) (2020)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [Generalized Self-Concordant Functions: A Recipe for Newton-Type Methods](https://mlsanity.com/paper/1703.04599/) (2017)
- [Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?](https://mlsanity.com/paper/2405.04230/) (2024)
- [Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamics](https://mlsanity.com/paper/2402.10680/) (2024)
- [Optical Flow Training under Limited Label Budget via Active Learning](https://mlsanity.com/paper/2203.05053/) (2022)
- [Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks](https://mlsanity.com/paper/2604.05230/) (2026)
- [Initialization-enhanced Physics-Informed Neural Network with Domain Decomposition (IDPINN)](https://mlsanity.com/paper/2406.03172/) (2024)
- [How Early Is Early Enough? Design-Dependent Observation-Window Sufficiency in Subscription Churn Prediction](https://mlsanity.com/paper/2607.00473/) (2026)
- [Quasi-Newton Methods: Superlinear Convergence Without Line Searches for Self-Concordant Functions](https://mlsanity.com/paper/1612.06965/) (2016)
- [Complex Physics-Informed Neural Network](https://mlsanity.com/paper/2502.04917/) (2025)
- [Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.21404/) (2025)
- [Convolution-weighting method for the physics-informed neural network: A Primal-Dual Optimization Perspective](https://mlsanity.com/paper/2506.19805/) (2025)
- [Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice](https://mlsanity.com/paper/2605.26559/) (2026)
- [Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting](https://mlsanity.com/paper/2606.18367/) (2026)
- [Regularization of Limited Memory Quasi-Newton Methods for Large-Scale Nonconvex Minimization](https://mlsanity.com/paper/1911.04584/) (2019)
- [TINNs: Time-Induced Neural Networks for Solving Time-Dependent PDEs](https://mlsanity.com/paper/2601.20361/) (2026)
- [Validation-Stage Combinatorial Fusion Analysis for Imbalanced Credit-Card Fraud Detection](https://mlsanity.com/paper/2606.10393/) (2026)
- [Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees](https://mlsanity.com/paper/2606.26432/) (2026)
- [Interpretable vs Learned Encoders for High-Cardinality Fraud Detection](https://mlsanity.com/paper/2607.00477/) (2026)
- [A Regularized Limited Memory BFGS method for Large-Scale Unconstrained Optimization and its Efficient Implementations](https://mlsanity.com/paper/2101.04413/) (2021)
- [Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm](https://mlsanity.com/paper/2602.08515/) (2026)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) often require high-accuracy quasi-Newton refinement to obtain reliable partial differential equation solutions, but their residual objectives can exhibit indefinite, nearly singular, and poorly scaled local curvature. Regularized quasi-Newton methods provide established mechanisms for stabilizing secant models, while self-concordant methods provide local-metric rules for curvature-dependent step selection. Building on these two lines of work, we propose SCORE, a self-concordance-inspired quasi-Newton method with decrement-coupled shifted secant geometry for PINN training. Its distinguishing mechanism is that a single quasi-Newton decrement computed from the learned inverse metric jointly determines a strong-Wolfe-tested candidate step and an adaptive shift used to define the next secant geometry. The shifted displacement represents the action of an averaged shifted metric along the accepted step, while requiring neither Hessian construction nor Hessian-vector products. Under a local spectral-equivalence condition, we show that the quasi-Newton decrement and candidate step remain comparable to their counterparts in a positive shifted metric, and recover the normalized self-concordant rule in the matched-metric case. Strong Wolfe acceptance, fallback line search, and standard curvature safeguards provide globalization without modifying the underlying PINN objective. Experiments on the viscous Burgers, Kuramoto--Sivashinsky, Korteweg--de Vries, and complex Ginzburg--Landau equations show that SCORE attains lower final errors than the tested BFGS and self-scaled Broyden baselines. The Burgers ablation further indicates that shifted curvature stabilization and decrement-based step selection make complementary contributions to high-accuracy refinement.