# Optimal Linear Baseline Models for Scientific Machine Learning

arXiv: https://arxiv.org/abs/2508.05831 · HTML: https://mlsanity.com/paper/2508.05831/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 110 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Quant GANs: Deep Generation of Financial Time Series](https://mlsanity.com/paper/1907.06673/) (2019)
- [Unsupervised Alignment of Embeddings with Wasserstein Procrustes](https://mlsanity.com/paper/1805.11222/) (2018)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Medical image denoising using convolutional denoising autoencoders](https://mlsanity.com/paper/1608.04667/) (2016)
- [Dimensionality Reduction for k-Means Clustering and Low Rank Approximation](https://mlsanity.com/paper/1410.6801/) (2014)
- [Learning-Based Low-Rank Approximations](https://mlsanity.com/paper/1910.13984/) (2019)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Solving Bayesian Inverse Problems via Variational Autoencoders](https://mlsanity.com/paper/1912.04212/) (2019)
- [Regularized linear autoencoders recover the principal components, eventually](https://mlsanity.com/paper/2007.06731/) (2020)
- [Optimal low-rank approximations of Bayesian linear inverse problems](https://mlsanity.com/paper/1407.3463/) (2014)
- [Learning Regularization Parameters of Inverse Problems via Deep Neural Networks](https://mlsanity.com/paper/2104.06594/) (2021)
- [Deep Neural Network Approach to Forward-Inverse Problems](https://mlsanity.com/paper/1907.12925/) (2019)
- [Subspace-Orbit Randomized Decomposition for Low-rank Matrix Approximation](https://mlsanity.com/paper/1804.00462/) (2018)
- [Paired Autoencoders for Likelihood-free Estimation in Inverse Problems](https://mlsanity.com/paper/2405.13220/) (2024)
- [A Paired Autoencoder Framework for Inverse Problems via Bayes Risk Minimization](https://mlsanity.com/paper/2501.14636/) (2025)
- [A Mechanism for Producing Aligned Latent Spaces with Autoencoders](https://mlsanity.com/paper/2106.15456/) (2021)
- [Rank Reduction Autoencoders](https://mlsanity.com/paper/2405.13980/) (2024)
- [Good Things Come in Pairs: Paired Autoencoders for Inverse Problems](https://mlsanity.com/paper/2505.06549/) (2025)
- [Solution Paths of Variational Regularization Methods for Inverse Problems](https://mlsanity.com/paper/1808.01783/) (2018)
- [Exact representation and efficient approximations of linear model predictive control laws via HardTanh type deep neural networks](https://mlsanity.com/paper/2401.05076/) (2024)
- [Variational Regularization in Inverse Problems and Machine Learning](https://mlsanity.com/paper/2112.04591/) (2021)

## Abstract (arXiv)

Across scientific domains, a fundamental challenge is to characterize and compute the mappings from underlying physical processes to observed signals and measurements. While nonlinear neural networks have achieved considerable success, they remain theoretically opaque, which hinders adoption in contexts where interpretability is paramount. In contrast, linear neural networks serve as a simple yet effective foundation for gaining insight into these complex relationships. In this work, we develop a unified theoretical framework for analyzing linear encoder-decoder architectures through the lens of Bayes risk minimization for solving data-driven scientific machine learning problems. We derive closed-form, rank-constrained linear and affine linear optimal mappings for forward modeling and inverse recovery tasks. Our results generalize existing formulations by accommodating rank-deficiencies in data, forward operators, and measurement processes. We validate our theoretical results by conducting numerical experiments on datasets from simple biomedical imaging, financial factor analysis, and simulations involving nonlinear fluid dynamics via the shallow water equations. This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.