# Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence

arXiv: https://arxiv.org/abs/2504.15487 · HTML: https://mlsanity.com/paper/2504.15487/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 77 references that are in the map.
- First posted on arXiv 21 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)
- [Deep transfer operator learning for partial differential equations under conditional shift](https://mlsanity.com/paper/2204.09810/) (2022)
- [Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations](https://mlsanity.com/paper/1709.00037/) (2017)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios](https://mlsanity.com/paper/2205.07731/) (2022)
- [Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution](https://mlsanity.com/paper/2304.07029/) (2023)
- [Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equations](https://mlsanity.com/paper/2212.04663/) (2022)
- [OceanNet: A principled neural operator-based digital twin for regional oceans](https://mlsanity.com/paper/2310.00813/) (2023)
- [Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow](https://mlsanity.com/paper/2206.03198/) (2022)
- [Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning](https://mlsanity.com/paper/2002.11167/) (2020)
- [SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition](https://mlsanity.com/paper/2211.08760/) (2022)
- [Analysis of three dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis](https://mlsanity.com/paper/2010.12060/) (2020)
- [Unsupervised Reservoir Computing for Solving Ordinary Differential Equations](https://mlsanity.com/paper/2108.11417/) (2021)
- [Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges](https://mlsanity.com/paper/2306.05014/) (2023)
- [Simultaneous emulation and downscaling with physically-consistent deep learning-based regional ocean emulators](https://mlsanity.com/paper/2501.05058/) (2025)

## Built on by (most cited first, 1 of 1)

- [VideoPDE: Unified Generative PDE Solving via Video Inpainting Diffusion Models](https://mlsanity.com/paper/2506.13754/) (2025)

## Abstract (arXiv)

Transfer learning (TL) is a powerful tool for enhancing the performance of neural networks (NNs) in applications such as weather and climate prediction and turbulence modeling. TL enables models to generalize to out-of-distribution data with minimal training data from the new system. In this study, we employ a 9-layer convolutional NN to predict the subgrid forcing in a two-layer ocean quasi-geostrophic system and examine which metrics best describe its performance and generalizability to unseen dynamical regimes. Fourier analysis of the NN kernels reveals that they learn low-pass, Gabor, and high-pass filters, regardless of whether the training data are isotropic or anisotropic. By analyzing the activation spectra, we identify why NNs fail to generalize without TL and how TL can overcome these limitations: the learned weights and biases from one dataset underestimate the out-of-distribution sample spectra as they pass through the network, leading to an underestimation of output spectra. By re-training only one layer with data from the target system, this underestimation is corrected, enabling the NN to produce predictions that match the target spectra. These findings are broadly applicable to data-driven parameterization of dynamical systems.