# Training Variation of Physically-Informed Deep Learning Models

arXiv: https://arxiv.org/abs/2510.03416 · HTML: https://mlsanity.com/paper/2510.03416/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Builds on (most cited first, 13 of 13)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies](https://mlsanity.com/paper/1906.00425/) (2019)
- [When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learning](https://mlsanity.com/paper/2203.16797/) (2022)
- [Deep learning for synthetic microstructure generation in a materials-by-design framework for heterogeneous energetic materials](https://mlsanity.com/paper/2004.04814/) (2020)
- [Improving the repeatability of deep learning models with Monte Carlo dropout](https://mlsanity.com/paper/2202.07562/) (2022)
- [Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics](https://mlsanity.com/paper/2206.01693/) (2022)
- [Uncertainty Prediction for Machine Learning Models of Material Properties](https://mlsanity.com/paper/2107.07997/) (2021)
- [Study and Observation of the Variation of Accuracies of KNN, SVM, LMNN, ENN Algorithms on Eleven Different Datasets from UCI Machine Learning Repository](https://mlsanity.com/paper/1809.06186/) (2018)
- [On the reproducibility of fully convolutional neural networks for modeling time-space evolving physical systems](https://mlsanity.com/paper/2105.05482/) (2021)

## Abstract (arXiv)

A successful deep learning network is highly dependent not only on the training dataset, but the training algorithm used to condition the network for a given task. The loss function, dataset, and tuning of hyperparameters all play an essential role in training a network, yet there is not much discussion on the reliability or reproducibility of a training algorithm. With the rise in popularity of physics-informed loss functions, this raises the question of how reliable one's loss function is in conditioning a network to enforce a particular boundary condition. Reporting the model variation is needed to assess a loss function's ability to consistently train a network to obey a given boundary condition, and provides a fairer comparison among different methods. In this work, a Pix2Pix network predicting the stress fields of high elastic contrast composites is used as a case study. Several different loss functions enforcing stress equilibrium are implemented, with each displaying different levels of variation in convergence, accuracy, and enforcing stress equilibrium across many training sessions. Suggested practices in reporting model variation are also shared.