# Diagnosing the conditional-mean barrier in scientific machine-learning surrogates

arXiv: https://arxiv.org/abs/2605.28076 · HTML: https://mlsanity.com/paper/2605.28076/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 12 of 12)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning](https://mlsanity.com/paper/2106.09004/) (2021)
- [Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models](https://mlsanity.com/paper/2108.12956/) (2021)
- [A Training-Free Conditional Diffusion Model for Learning Stochastic Dynamical Systems](https://mlsanity.com/paper/2410.03108/) (2024)
- [Semi Conditional Variational Auto-Encoder for Flow Reconstruction and Uncertainty Quantification from Limited Observations](https://mlsanity.com/paper/2007.09644/) (2020)
- [DynGMA: a robust approach for learning stochastic differential equations from data](https://mlsanity.com/paper/2402.14475/) (2024)

## Abstract (arXiv)

Many prediction tasks in computational science and engineering become one-to-many after coarse graining and partial observation. In such settings, deterministic surrogates trained by squared loss may learn a well-defined mathematical object, the conditional mean, while still missing the task-relevant variability in the underlying conditional law. In this work, we formulate this limitation as the conditional-mean barrier and develop a diagnostic framework for identifying it in fitted scientific machine-learning surrogates. The framework combines residual-feature orthogonality and effect-size diagnostics to distinguish deterministic underfitting from irreducible conditional variability. We also make explicit a simple consequence of paired squared loss: stochastic outputs do not by themselves overcome the barrier, because the objective penalizes model variance and drives the predictor back to the conditional mean. The diagnosis therefore yields a modeling prescription: when residual variability matters, the loss must score richer features of the conditional law rather than a point prediction. Reproducible numerical studies on a controlled two-branch law and a two-scale Lorenz-96 closure problem show how the diagnostic identifies the barrier, how deterministic closures can suppress collective fluctuation statistics in rollout, and how a minimal likelihood-based stochastic-scale model can recover substantially more variability.