# A Polynomial-Based Approach for Architectural Design and Learning with Deep Neural Networks

arXiv: https://arxiv.org/abs/1905.10457 · HTML: https://mlsanity.com/paper/1905.10457/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2019.

## Built on by (most cited first, 4 of 4)

- [Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks](https://mlsanity.com/paper/2404.03761/) (2024)
- [CAS4DL: Christoffel Adaptive Sampling for function approximation via Deep Learning](https://mlsanity.com/paper/2208.12190/) (2022)
- [Machine Learning Optimized Orthogonal Basis Piecewise Polynomial Approximation](https://mlsanity.com/paper/2403.08579/) (2024)
- [Data-driven discovery of Green's functions](https://mlsanity.com/paper/2210.16016/) (2022)

## Abstract (arXiv)

In this effort we propose a novel approach for reconstructing multivariate functions from training data, by identifying both a suitable network architecture and an initialization using polynomial-based approximations. Training deep neural networks using gradient descent can be interpreted as moving the set of network parameters along the loss landscape in order to minimize the loss functional. The initialization of parameters is important for iterative training methods based on descent. Our procedure produces a network whose initial state is a polynomial representation of the training data. The major advantage of this technique is from this initialized state the network may be improved using standard training procedures. Since the network already approximates the data, training is more likely to produce a set of parameters associated with a desirable local minimum. We provide the details of the theory necessary for constructing such networks and also consider several numerical examples that reveal our approach ultimately produces networks which can be effectively trained from our initialized state to achieve an improved approximation for a large class of target functions.