# Multi-Modal Learning-based Reconstruction of High-Resolution Spatial Wind Speed Fields

arXiv: https://arxiv.org/abs/2312.08933 · HTML: https://mlsanity.com/paper/2312.08933/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 73 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)
- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [Stochastic Super-Resolution for Downscaling Time-Evolving Atmospheric Fields with a Generative Adversarial Network](https://mlsanity.com/paper/2005.10374/) (2020)
- [Machine Learning for the Geosciences: Challenges and Opportunities](https://mlsanity.com/paper/1711.04708/) (2017)
- [Learning Variational Data Assimilation Models and Solvers](https://mlsanity.com/paper/2007.12941/) (2020)
- [SwinRDM: Integrate SwinRNN with Diffusion Model towards High-Resolution and High-Quality Weather Forecasting](https://mlsanity.com/paper/2306.03110/) (2023)
- [Learning The Invisible: A Hybrid Deep Learning-Shearlet Framework for Limited Angle Computed Tomography](https://mlsanity.com/paper/1811.04602/) (2018)
- [Diffusion Models for High-Resolution Solar Forecasts](https://mlsanity.com/paper/2302.00170/) (2023)
- [Deep Learning-Based Solvability of Underdetermined Inverse Problems in Medical Imaging](https://mlsanity.com/paper/2001.01432/) (2020)
- [Deep Diffusion Models for Seismic Processing](https://mlsanity.com/paper/2207.10451/) (2022)
- [4DVarNet-SSH: end-to-end learning of variational interpolation schemes for nadir and wide-swath satellite altimetry](https://mlsanity.com/paper/2211.05904/) (2022)
- [Joint learning of variational representations and solvers for inverse problems with partially-observed data](https://mlsanity.com/paper/2006.03653/) (2020)
- [Training neural mapping schemes for satellite altimetry with simulation data](https://mlsanity.com/paper/2309.14350/) (2023)
- [Neural SPDE solver for uncertainty quantification in high-dimensional space-time dynamics](https://mlsanity.com/paper/2311.01783/) (2023)

## Abstract (arXiv)

Wind speed at sea surface is a key quantity for a variety of scientific applications and human activities. Due to the non-linearity of the phenomenon, a complete description of such variable is made infeasible on both the small scale and large spatial extents. Methods relying on Data Assimilation techniques, despite being the state-of-the-art for Numerical Weather Prediction, can not provide the reconstructions with a spatial resolution that can compete with satellite imagery. In this work we propose a framework based on Variational Data Assimilation and Deep Learning concepts. This framework is applied to recover rich-in-time, high-resolution information on sea surface wind speed. We design our experiments using synthetic wind data and different sampling schemes for high-resolution and low-resolution versions of original data to emulate the real-world scenario of spatio-temporally heterogeneous observations. Extensive numerical experiments are performed to assess systematically the impact of low and high-resolution wind fields and in-situ observations on the model reconstruction performance. We show that in-situ observations with richer temporal resolution represent an added value in terms of the model reconstruction performance. We show how a multi-modal approach, that explicitly informs the model about the heterogeneity of the available observations, can improve the reconstruction task by exploiting the complementary information in spatial and local point-wise data. To conclude, we propose an analysis to test the robustness of the chosen framework against phase delay and amplitude biases in low-resolution data and against interruptions of in-situ observations supply at evaluation time