# Approximation by non-symmetric networks for cross-domain learning

arXiv: https://arxiv.org/abs/2305.03890 · HTML: https://mlsanity.com/paper/2305.03890/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 6 May 2023.

## Builds on (most cited first, 15 of 15)

- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [To understand deep learning we need to understand kernel learning](https://mlsanity.com/paper/1802.01396/) (2018)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Deep vs. shallow networks : An approximation theory perspective](https://mlsanity.com/paper/1608.03287/) (2016)
- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [Understanding neural networks with reproducing kernel Banach spaces](https://mlsanity.com/paper/2109.09710/) (2021)
- [Deep nets for local manifold learning](https://mlsanity.com/paper/1607.07110/) (2016)
- [Kernel based analysis of massive data](https://mlsanity.com/paper/2003.13226/) (2020)
- [Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions](https://mlsanity.com/paper/1709.08174/) (2017)
- [Dimension independent bounds for general shallow networks](https://mlsanity.com/paper/1908.09880/) (2019)
- [On the relationship between multivariate splines and infinitely-wide neural networks](https://mlsanity.com/paper/2302.03459/) (2023)
- [A witness function based construction of discriminative models using Hermite polynomials](https://mlsanity.com/paper/1901.02975/) (2019)
- [An analysis of training and generalization errors in shallow and deep networks](https://mlsanity.com/paper/1802.06266/) (2018)
- [Bigeometric Organization of Deep Nets](https://mlsanity.com/paper/1507.00220/) (2015)

## Built on by (most cited first, 1 of 1)

- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)

## Abstract (arXiv)

For the past 30 years or so, machine learning has stimulated a great deal of research in the study of approximation capabilities (expressive power) of a multitude of processes, such as approximation by shallow or deep neural networks, radial basis function networks, and a variety of kernel based methods. Motivated by applications such as invariant learning, transfer learning, and synthetic aperture radar imaging, we initiate in this paper a general approach to study the approximation capabilities of kernel based networks using non-symmetric kernels. While singular value decomposition is a natural instinct to study such kernels, we consider a more general approach to include the use of a family of kernels, such as generalized translation networks (which include neural networks and translation invariant kernels as special cases) and rotated zonal function kernels. Naturally, unlike traditional kernel based approximation, we cannot require the kernels to be positive definite. In particular, we obtain estimates on the accuracy of uniform approximation of functions in a Sobolev class by ReLUr networks when r is not necessarily an integer. Our general results apply to the approximation of functions with small smoothness compared to the dimension of the input space.