On finite-dimensional encoding/decoding theorems for neural operators

Recently, versions of neural networks with infinite-dimensional affine operators inside the computational units (``neural operator'' networks) have been applied to learn solutions to differential equations. To enable practical computations, one employs finite-dimensional encoding/decoding theorems of the following kind: every continuous mapping f between function spaces E and F is approximated in the topology of uniform convergence on compacta by continuous mappings factoring through two finite dimensional Banach spaces. Such a result is known (Kovachki et al., 2023) for E,F being Banach spaces having the approximation property. We point out that the result needs no assumptions on E,F whatsoever and remains true not only for all normed spaces, but for arbitrary locally convex spaces as well. At the same time, an analogous result for Ck-smooth mappings and the Ck compact open topology, k≥ 1, holds if and only if the space E has the approximation property. This analysis may be useful already because non-normable locally convex function spaces are common in the theory of differential equations, the main field of applications for the emerging theory.