# Globally injective and bijective neural operators

arXiv: https://arxiv.org/abs/2306.03982 · HTML: https://mlsanity.com/paper/2306.03982/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [The Reversible Residual Network: Backpropagation Without Storing Activations](https://mlsanity.com/paper/1707.04585/) (2017)
- [NETT: Solving Inverse Problems with Deep Neural Networks](https://mlsanity.com/paper/1803.00092/) (2018)
- [Generative Models as Distributions of Functions](https://mlsanity.com/paper/2102.04776/) (2021)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [Score-based Diffusion Models in Function Space](https://mlsanity.com/paper/2302.07400/) (2023)
- [Understanding Variational Inference in Function-Space](https://mlsanity.com/paper/2011.09421/) (2020)
- [Universal approximation property of invertible neural networks](https://mlsanity.com/paper/2204.07415/) (2022)
- [Non-Euclidean Universal Approximation](https://mlsanity.com/paper/2006.02341/) (2020)
- [Globally Injective ReLU Networks](https://mlsanity.com/paper/2006.08464/) (2020)
- [IAE-Net: Integral Autoencoders for Discretization-Invariant Learning](https://mlsanity.com/paper/2203.05142/) (2022)
- [Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows](https://mlsanity.com/paper/2110.04227/) (2021)
- [Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows](https://mlsanity.com/paper/2007.07985/) (2020)
- [Continuous Generative Neural Networks: A Wavelet-Based Architecture in Function Spaces](https://mlsanity.com/paper/2205.14627/) (2022)

## Built on by (most cited first, 13 of 13)

- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Transformers are Universal In-context Learners](https://mlsanity.com/paper/2408.01367/) (2024)
- [Mixtures of Neural Operators Reduce Active Complexity in Operator Learning](https://mlsanity.com/paper/2404.09101/) (2024)
- [Universal Functional Regression with Neural Operator Flows](https://mlsanity.com/paper/2404.02986/) (2024)
- [A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theory](https://mlsanity.com/paper/2510.25379/) (2025)
- [Injectivity capacity of ReLU gates](https://mlsanity.com/paper/2410.20646/) (2024)
- [New universal operator approximation theorem for encoder-decoder architectures](https://mlsanity.com/paper/2503.24092/) (2025)
- [Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning](https://mlsanity.com/paper/2605.22724/) (2026)
- [Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networks](https://mlsanity.com/paper/2604.01961/) (2026)
- [Complexity of Injectivity and Verification of ReLU Neural Networks](https://mlsanity.com/paper/2405.19805/) (2024)
- [Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEs](https://mlsanity.com/paper/2410.14788/) (2024)
- [Can neural operators always be continuously discretized?](https://mlsanity.com/paper/2412.03393/) (2024)
- [NeuralChaos: Optimal Adapted Approximation of Square Integrable Predictable Processes](https://mlsanity.com/paper/2607.14361/) (2026)

## Abstract (arXiv)

Recently there has been great interest in operator learning, where networks learn operators between function spaces from an essentially infinite-dimensional perspective. In this work we present results for when the operators learned by these networks are injective and surjective. As a warmup, we combine prior work in both the finite-dimensional ReLU and operator learning setting by giving sharp conditions under which ReLU layers with linear neural operators are injective. We then consider the case the case when the activation function is pointwise bijective and obtain sufficient conditions for the layer to be injective. We remark that this question, while trivial in the finite-rank case, is subtler in the infinite-rank case and is proved using tools from Fredholm theory. Next, we prove that our supplied injective neural operators are universal approximators and that their implementation, with finite-rank neural networks, are still injective. This ensures that injectivity is not `lost' in the transcription from analytical operators to their finite-rank implementation with networks. Finally, we conclude with an increase in abstraction and consider general conditions when subnetworks, which may be many layers deep, are injective and surjective and provide an exact inversion from a `linearization.' This section uses general arguments from Fredholm theory and Leray-Schauder degree theory for non-linear integral equations to analyze the mapping properties of neural operators in function spaces. These results apply to subnetworks formed from the layers considered in this work, under natural conditions. We believe that our work has applications in Bayesian UQ where injectivity enables likelihood estimation and in inverse problems where surjectivity and injectivity corresponds to existence and uniqueness, respectively.