# Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning

arXiv: https://arxiv.org/abs/2002.11167 · HTML: https://mlsanity.com/paper/2002.11167/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 17 of 17)

- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Model scale versus domain knowledge in statistical forecasting of chaotic systems](https://mlsanity.com/paper/2303.08011/) (2023)
- [Learning Spatiotemporal Chaos Using Next-Generation Reservoir Computing](https://mlsanity.com/paper/2203.13294/) (2022)
- [Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow](https://mlsanity.com/paper/2206.03198/) (2022)
- [Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers](https://mlsanity.com/paper/2103.09360/) (2021)
- [Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods](https://mlsanity.com/paper/2103.10935/) (2021)
- [Multiscale Neural Operator: Learning Fast and Grid-independent PDE Solvers](https://mlsanity.com/paper/2207.11417/) (2022)
- [Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges](https://mlsanity.com/paper/2306.05014/) (2023)
- [A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization](https://mlsanity.com/paper/2208.09104/) (2022)
- [Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case](https://mlsanity.com/paper/2110.00546/) (2021)
- [Using Probabilistic Machine Learning to Better Model Temporal Patterns in Parameterizations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2203.14814/) (2022)
- [Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles](https://mlsanity.com/paper/2302.09748/) (2023)
- [Minimum Reduced-Order Models via Causal Inference](https://mlsanity.com/paper/2407.00271/) (2024)
- [Semi-automatic tuning of coupled climate models with multiple intrinsic timescales: lessons learned from the Lorenz96 model](https://mlsanity.com/paper/2208.06243/) (2022)
- [Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence](https://mlsanity.com/paper/2504.15487/) (2025)
- [Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation](https://mlsanity.com/paper/2210.04001/) (2022)
- [Using Echo-State Networks to Reproduce Rare Events in Chaotic Systems](https://mlsanity.com/paper/2505.16208/) (2025)

## Abstract (arXiv)

To make weather/climate modeling computationally affordable, small-scale processes are usually represented in terms of the large-scale, explicitly-resolved processes using physics-based or semi-empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super-parameterization (SP), which involves integrating the equations of small-scale processes on high-resolution grids embedded within the low-resolution grids of large-scale processes. Recently, studies have used machine learning (ML) to develop data-driven parameterization (DD-P) schemes. Here, we propose a new approach, data-driven SP (DD-SP), in which the equations of the small-scale processes are integrated data-drivenly using ML methods such as recurrent neural networks. Employing multi-scale Lorenz 96 systems as testbed, we compare the cost and accuracy (in terms of both short-term prediction and long-term statistics) of parameterized low-resolution (LR), SP, DD-P, and DD-SP models. We show that with the same computational cost, DD-SP substantially outperforms LR, and is better than DD-P, particularly when scale separation is lacking. DD-SP is much cheaper than SP, yet its accuracy is the same in reproducing long-term statistics and often comparable in short-term forecasting. We also investigate generalization, finding that when models trained on data from one system are applied to a system with different forcing (e.g., more chaotic), the models often do not generalize, particularly when the short-term prediction accuracy is examined. But we show that transfer-learning, which involves re-training the data-driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD-SP and transfer-learning in climate/weather modeling and the expected challenges are discussed.