mlsanityData to 7 Oct 2026

Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning

arXiv 2401.10490 ↗First posted 19 Jan 2024cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this context, seeks to extract these physical processes from empirical data, which is challenging due to the infinite or high dimensionality of data. An integral component in addressing this challenge is model reduction, which reduces both the data dimensionality and problem size. In this paper, we utilize low-dimensional nonlinear structures in model reduction by investigating Auto-Encoder-based Neural Network (AENet). AENet first learns the latent variables of the input data and then learns the transformation from these latent variables to corresponding output data. Our numerical experiments validate the ability of AENet to accurately learn the solution operator of nonlinear partial differential equations. Furthermore, we establish a mathematical and statistical estimation theory that analyzes the generalization error of AENet. Our theoretical framework shows that the sample complexity of training AENet is intricately tied to the intrinsic dimension of the modeled process, while also demonstrating the remarkable resilience of AENet to noise.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 31 of its 69 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

31 of 69 references.

References · 31

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  3. 2019An Introduction to Variational Autoencoderscited by 997
  4. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
  5. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  6. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  7. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  8. 2016Error bounds for approximations with deep ReLU networkscited by 567
  9. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  10. 2021Neural Operator: Learning Maps Between Function Spacescited by 413
  11. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  12. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 290
19 more
  1. 2020Model Reduction and Neural Networks for Parametric PDEscited by 257
  2. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  3. 2020Deep Network Approximation for Smooth Functionscited by 163
  4. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  5. 2017Linearly-Recurrent Autoencoder Networks for Learning Dynamicscited by 142
  6. 2020A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEscited by 105
  7. 2022NOMAD: Nonlinear Manifold Decoders for Operator Learningcited by 81
  8. 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
  9. 2018Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systemscited by 54
  10. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  11. 2020Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEscited by 40
  12. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  13. 2018SwitchNet: a neural network model for forward and inverse scattering problemscited by 35
  14. 2019Chart Auto-Encoders for Manifold Structured Datacited by 32
  15. 2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
  16. 2021A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equationscited by 28
  17. 2023Learning in latent spaces improves the predictive accuracy of deep neural operatorscited by 19
  18. 2020Efficient nonlinear manifold reduced order modelcited by 17
  19. 2023Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustnesscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 3.

12 papers.

  1. 2024Convergence Analysis of Flow Matching in Latent Space with Transformerscited by 16
  2. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 13
  3. 2024A practical existence theorem for reduced order models based on convolutional autoencoderscited by 9
  4. 2024Controlling Statistical, Discretization, and Truncation Errors in Learning Fourier Linear Operatorscited by 4
  5. 2024Spherical Analysis of Learning Nonlinear Functionalscited by 3
  6. 2025Cauchy Random Features for Operator Learning in Sobolev Spacecited by 3
  7. 2025Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guaranteescited by 3
  8. 2025Regularized Random Fourier Features and Finite Element Reconstruction for Operator Learning in Sobolev Spacecited by 1
  9. 2026Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learningcited by 1
  10. 2025Data-Driven Model Reduction using WeldNet: Windowed Encoders for Learning Dynamicscited by 0
  11. 2026Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimensioncited by 0
  12. 2026Neural Scaling Laws of Transformer Operator Networkcited by 0