Noncompact uniform universal approximation

The universal approximation theorem is generalised to uniform convergence on the (noncompact) input space ℝn. All continuous functions that vanish at infinity can be uniformly approximated by neural networks with one hidden layer, for all activation functions varphi that are continuous, nonpolynomial, and asymptotically polynomial at ±∞. When varphi is moreover bounded, we exactly determine which functions can be uniformly approximated by neural networks, with the following unexpected results. Let overlineN_varphil(ℝn) denote the vector space of functions that are uniformly approximable by neural networks with l hidden layers and n inputs. For all n and all l≥2, overlineN_varphil(ℝn) turns out to be an algebra under the pointwise product. If the left limit of varphi differs from its right limit (for instance, when varphi is sigmoidal) the algebra overlineN_varphil(ℝn) (l≥2) is independent of varphi and l, and equals the closed span of products of sigmoids composed with one-dimensional projections. If the left limit of varphi equals its right limit, overlineN_varphil(ℝn) (l≥1) equals the (real part of the) commutative resolvent algebra, a C*-algebra which is used in mathematical approaches to quantum theory. In the latter case, the algebra is independent of l≥1, whereas in the former case overlineN_varphi2(ℝn) is strictly bigger than overlineN_varphi1(ℝn).