# Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacology

arXiv: https://arxiv.org/abs/2504.07379 · HTML: https://mlsanity.com/paper/2504.07379/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 58 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Builds on (most cited first, 22 of 22)

- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability](https://mlsanity.com/paper/2103.00065/) (2021)
- [Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation](https://mlsanity.com/paper/2405.07200/) (2024)
- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [Adaptive Gradient Methods at the Edge of Stability](https://mlsanity.com/paper/2207.14484/) (2022)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?](https://mlsanity.com/paper/2405.04230/) (2024)
- [AI-Aristotle: A Physics-Informed framework for Systems Biology Gray-Box Identification](https://mlsanity.com/paper/2310.01433/) (2023)
- [State-space models are accurate and efficient neural operators for dynamical systems](https://mlsanity.com/paper/2409.03231/) (2024)
- [Systems Biology: Identifiability analysis and parameter identification via systems-biology informed neural networks](https://mlsanity.com/paper/2202.01723/) (2022)
- [An Approach to Symbolic Regression Using Feyn](https://mlsanity.com/paper/2104.05417/) (2021)
- [Optimistix: modular optimisation in JAX and Equinox](https://mlsanity.com/paper/2402.09983/) (2024)
- [A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data](https://mlsanity.com/paper/2305.10706/) (2023)
- [A Greedy Search Tree Heuristic for Symbolic Regression](https://mlsanity.com/paper/1801.01807/) (2018)
- [Partial observations and conservation laws: Grey-box modeling in biotechnology and optogenetics](https://mlsanity.com/paper/1909.04234/) (2019)
- [Physics-Informed Neural Networks and Extensions](https://mlsanity.com/paper/2408.16806/) (2024)
- [Structured Quasi-Newton Methods for Optimization with Orthogonality Constraints](https://mlsanity.com/paper/1809.00452/) (2018)
- [CMINNs: Compartment Model Informed Neural Networks -- Unlocking Drug Dynamics](https://mlsanity.com/paper/2409.12998/) (2024)
- [Black and Gray Box Learning of Amplitude Equations: Application to Phase Field Systems](https://mlsanity.com/paper/2207.03954/) (2022)
- [Training Neural Networks in Single vs Double Precision](https://mlsanity.com/paper/2209.07219/) (2022)

## Built on by (most cited first, 10 of 10)

- [ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms](https://mlsanity.com/paper/2512.03476/) (2025)
- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [Physics-Informed Machine Learning in Biomedical Science and Engineering](https://mlsanity.com/paper/2510.05433/) (2025)
- [Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning](https://mlsanity.com/paper/2603.23854/) (2026)
- [Making Gaussian Kolmogorov-Arnold Networks Reliable and Accurate](https://mlsanity.com/paper/2604.21174/) (2026)
- [Empirical Stability Analysis of Kolmogorov-Arnold Networks in Hard-Constrained Recurrent Physics-Informed Discovery](https://mlsanity.com/paper/2602.09988/) (2026)
- [Partition-of-Unity Gaussian Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2604.23599/) (2026)
- [The Enhanced Physics-Informed Kolmogorov-Arnold Networks: Applications of Newton's Laws in Financial Deep Reinforcement Learning (RL) Algorithms](https://mlsanity.com/paper/2602.01388/) (2026)
- [Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit](https://mlsanity.com/paper/2605.08352/) (2026)
- [OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems](https://mlsanity.com/paper/2606.19145/) (2026)

## Abstract (arXiv)

Physics-Informed Kolmogorov-Arnold Networks (PIKANs) are gaining attention as an effective counterpart to the original multilayer perceptron-based Physics-Informed Neural Networks (PINNs). Both representation models can address inverse problems and facilitate gray-box system identification. However, a comprehensive understanding of their performance in terms of accuracy and speed remains underexplored. In particular, we introduce a modified PIKAN architecture, tanh-cPIKAN, which is based on Chebyshev polynomials for parametrization of the univariate functions with an extra nonlinearity for enhanced performance. We then present a systematic investigation of how choices of the optimizer, representation, and training configuration influence the performance of PINNs and PIKANs in the context of systems pharmacology modeling. We benchmark a wide range of first-order, second-order, and hybrid optimizers, including various learning rate schedulers. We use the new Optax library to identify the most effective combinations for learning gray-boxes under ill-posed, non-unique, and data-sparse conditions. We examine the influence of model architecture (MLP vs. KAN), numerical precision (single vs. double), the need for warm-up phases for second-order methods, and sensitivity to the initial learning rate. We also assess the optimizer scalability for larger models and analyze the trade-offs introduced by JAX in terms of computational efficiency and numerical accuracy. Using two representative systems pharmacology case studies - a pharmacokinetics model and a chemotherapy drug-response model - we offer practical guidance on selecting optimizers and representation models/architectures for robust and efficient gray-box discovery. Our findings provide actionable insights for improving the training of physics-informed networks in biomedical applications and beyond.