# Into the danger zone: stable extrapolation in high-dimensional function and operator learning

arXiv: https://arxiv.org/abs/2609.36709 · HTML: https://mlsanity.com/paper/2609.36709/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 202 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Out of Distribution Generalization in Machine Learning](https://mlsanity.com/paper/2103.02667/) (2021)
- [The Parametric Complexity of Operator Learning](https://mlsanity.com/paper/2306.15924/) (2023)
- [Defining Foundation Models for Computational Science: A Call for Clarity and Rigor](https://mlsanity.com/paper/2505.22904/) (2025)
- [The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures](https://mlsanity.com/paper/2410.23440/) (2024)
- [Bridging the Gap Between Approximation and Learning via Optimal Approximation by ReLU MLPs of Maximal Regularity](https://mlsanity.com/paper/2409.12335/) (2024)
- [Test-time Generalization for Physics through Neural Operator Splitting](https://mlsanity.com/paper/2602.00884/) (2026)
- [Understanding and Mitigating Extrapolation Failures in Physics-Informed Neural Networks](https://mlsanity.com/paper/2306.09478/) (2023)
- [SIMSHIFT: A Benchmark for Adapting Neural Surrogates to Distribution Shifts](https://mlsanity.com/paper/2506.12007/) (2025)
- [Performance of Neural and Polynomial Operator Surrogates](https://mlsanity.com/paper/2604.00689/) (2026)
- [Diagnosing Failure Modes of Neural Operators Across Diverse PDE Families](https://mlsanity.com/paper/2601.11428/) (2026)
- [Optimal Recovery Meets Minimax Estimation](https://mlsanity.com/paper/2502.17671/) (2025)
- [A short tour of operator learning theory: Convergence rates, statistical limits, and open questions](https://mlsanity.com/paper/2603.00819/) (2026)
- [Learning where to learn: Training data distribution optimization for scientific machine learning](https://mlsanity.com/paper/2505.21626/) (2025)
- [Out-of-distribution generalization of deep-learning surrogates for 2D PDE-generated dynamics in the small-data regime](https://mlsanity.com/paper/2601.08404/) (2026)
- [Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces](https://mlsanity.com/paper/2606.17419/) (2026)
- [Do Physics Foundation Models Learn Generalizable Physics? A Bias-Aware Benchmark Across Physical Regimes and Distribution Shifts](https://mlsanity.com/paper/2605.29283/) (2026)
- [Is Zero-Shot Super-Resolution Possible in Operator Learning?](https://mlsanity.com/paper/2606.00296/) (2026)
- [Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation](https://mlsanity.com/paper/2606.26705/) (2026)

## Abstract (arXiv)

Out-of-distribution (OOD) generalization is a central challenge in scientific machine learning. We study regression problems in which the test distribution differs from the training distribution and ask: under what assumptions on the target function or operator is stable extrapolation possible, and how far beyond the training domain can one extrapolate? Existing theory controls the test error through additive penalties measuring the discrepancy between the training and test distributions. Such guarantees show robustness to small distribution shifts, but can be very pessimistic in comparison to OOD performance observed empirically. We identify classes of holomorphic functions and operators for which the OOD generalization error converges at algebraic rates even in the presence of large distribution shifts. This phenomenon stems from the increasing smoothness of higher-index coordinates, leading to what we term a `blessing of high dimensionality'. For learning with either polynomials, deep neural networks or deep neural operators, we derive explicit rates for arbitrary test measures supported on suitable domains and quantify how the admissible domain depends on the underlying regularity of the function or operator. Our extrapolation guarantees are independent of the test distribution, depending only on its support. We also present a series of numerical experiments across a range of functions and operators that support the main theoretical findings.