# Function Extrapolation with Neural Networks and Its Application for Manifolds

arXiv: https://arxiv.org/abs/2405.10563 · HTML: https://mlsanity.com/paper/2405.10563/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 10 of 10)

- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [Regularized Nonlinear Acceleration](https://mlsanity.com/paper/1606.04133/) (2016)
- [Riemannian accelerated gradient methods via extrapolation](https://mlsanity.com/paper/2208.06619/) (2022)
- [On the Convergence of Stochastic Gradient Descent in Low-precision Number Formats](https://mlsanity.com/paper/2301.01651/) (2023)
- [Periodic Extrapolative Generalisation in Neural Networks](https://mlsanity.com/paper/2209.10280/) (2022)

## Built on by (most cited first, 1 of 1)

- [Just One Layer Norm Guarantees Stable Extrapolation](https://mlsanity.com/paper/2505.14512/) (2025)

## Abstract (arXiv)

This paper addresses the problem of accurately estimating a function on one domain when only its discrete samples are available on another domain. To answer this challenge, we utilize a neural network, which we train to incorporate prior knowledge of the function. In addition, by carefully analyzing the problem, we obtain a bound on the error over the extrapolation domain and define a condition number for this problem that quantifies the level of difficulty of the setup. Compared to other machine learning methods that provide time series prediction, such as transformers, our approach is suitable for setups where the interpolation and extrapolation regions are general subdomains and, in particular, manifolds. In addition, our construction leads to an improved loss function that helps us boost the accuracy and robustness of our neural network. We conduct comprehensive numerical tests and comparisons of our extrapolation versus standard methods. The results illustrate the effectiveness of our approach in various scenarios.