# Learning About Structural Errors in Models of Complex Dynamical Systems

arXiv: https://arxiv.org/abs/2401.00035 · HTML: https://mlsanity.com/paper/2401.00035/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 131 references that are in the map.
- First posted on arXiv 29 Dec 2023.

## Builds on (most cited first, 25 of 25)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [Neural Lander: Stable Drone Landing Control using Learned Dynamics](https://mlsanity.com/paper/1811.08027/) (2018)
- [Sparse learning of stochastic dynamic equations](https://mlsanity.com/paper/1712.02432/) (2017)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations](https://mlsanity.com/paper/1709.00037/) (2017)
- [Combining data assimilation and machine learning to infer unresolved scale parametrisation](https://mlsanity.com/paper/2009.04318/) (2020)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [nPINNs: nonlocal Physics-Informed Neural Networks for a parametrized nonlocal universal Laplacian operator. Algorithms and Applications](https://mlsanity.com/paper/2004.04276/) (2020)
- [Deep Neural Networks for Data-Driven Turbulence Models](https://mlsanity.com/paper/1806.04482/) (2018)
- [Auto-differentiable Ensemble Kalman Filters](https://mlsanity.com/paper/2107.07687/) (2021)
- [Model Reduction with Memory and the Machine Learning of Dynamical Systems](https://mlsanity.com/paper/1808.04258/) (2018)
- [Ensemble Kalman Methods: A Mean Field Perspective](https://mlsanity.com/paper/2209.11371/) (2022)
- [Inverse Problems and Data Assimilation](https://mlsanity.com/paper/1810.06191/) (2018)
- [Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series](https://mlsanity.com/paper/2111.13037/) (2021)
- [Approximate Bayes learning of stochastic differential equations](https://mlsanity.com/paper/1702.05390/) (2017)
- [Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods](https://mlsanity.com/paper/2104.03384/) (2021)
- [Iterative Updating of Model Error for Bayesian Inversion](https://mlsanity.com/paper/1707.04246/) (2017)
- [Embedded Model Error Representation for Bayesian Model Calibration](https://mlsanity.com/paper/1801.06768/) (2018)
- [Ensemble Kalman Methods With Constraints](https://mlsanity.com/paper/1901.05668/) (2019)
- [Identification of the diffusion parameter in nonlocal steady diffusion problems](https://mlsanity.com/paper/1310.2558/) (2013)

## Built on by (most cited first, 13 of 13)

- [Data-Driven Stochastic Closure Modeling via Conditional Diffusion Model and Neural Operator](https://mlsanity.com/paper/2408.02965/) (2024)
- [Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features](https://mlsanity.com/paper/2407.00584/) (2024)
- [CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation](https://mlsanity.com/paper/2410.20072/) (2024)
- [CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation](https://mlsanity.com/paper/2404.06749/) (2024)
- [Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network](https://mlsanity.com/paper/2507.08749/) (2025)
- [Active Learning of Model Discrepancy with Bayesian Experimental Design](https://mlsanity.com/paper/2502.05372/) (2025)
- [Stochastic and Non-local Closure Modeling for Nonlinear Dynamical Systems via Latent Score-based Generative Models](https://mlsanity.com/paper/2506.20771/) (2025)
- [Deep learning for model correction of dynamical systems with data scarcity](https://mlsanity.com/paper/2410.17913/) (2024)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [Structural Constraints for Physics-augmented Learning](https://mlsanity.com/paper/2410.05507/) (2024)
- [Synergizing Transport-Based Generative Models and Latent Geometry for Stochastic Closure Modeling](https://mlsanity.com/paper/2602.17089/) (2026)
- [Hybrid twinning using PBDW and DeepONet for the effective state estimation and prediction on partially known systems](https://mlsanity.com/paper/2512.11834/) (2025)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)

## Abstract (arXiv)

Complex dynamical systems are notoriously difficult to model because some degrees of freedom (e.g., small scales) may be computationally unresolvable or are incompletely understood, yet they are dynamically important. For example, the small scales of cloud dynamics and droplet formation are crucial for controlling climate, yet are unresolvable in global climate models. Semi-empirical closure models for the effects of unresolved degrees of freedom often exist and encode important domain-specific knowledge. Building on such closure models and correcting them through learning the structural errors can be an effective way of fusing data with domain knowledge. Here we describe a general approach, principles, and algorithms for learning about structural errors. Key to our approach is to include structural error models inside the models of complex systems, for example, in closure models for unresolved scales. The structural errors then map, usually nonlinearly, to observable data. As a result, however, mismatches between model output and data are only indirectly informative about structural errors, due to a lack of labeled pairs of inputs and outputs of structural error models. Additionally, derivatives of the model may not exist or be readily available. We discuss how structural error models can be learned from indirect data with derivative-free Kalman inversion algorithms and variants, how sparsity constraints enforce a "do no harm" principle, and various ways of modeling structural errors. We also discuss the merits of using non-local and/or stochastic error models. In addition, we demonstrate how data assimilation techniques can assist the learning about structural errors in non-ergodic systems. The concepts and algorithms are illustrated in two numerical examples based on the Lorenz-96 system and a human glucose-insulin model.