mlsanityData to 14 Oct 2026

Efficient deep data assimilation with sparse observations and time-varying sensors

arXiv 2310.16187 ↗First posted 24 Oct 2023cs.LG math-ph math.MPcited by 14

Not on the map. Why?

Abstract (arXiv)

Variational Data Assimilation (DA) has been broadly used in engineering problems for field reconstruction and prediction by performing a weighted combination of multiple sources of noisy data. In recent years, the integration of deep learning (DL) techniques in DA has shown promise in improving the efficiency and accuracy in high-dimensional dynamical systems. Nevertheless, existing deep DA approaches face difficulties in dealing with unstructured observation data, especially when the placement and number of sensors are dynamic over time. We introduce a novel variational DA scheme, named Voronoi-tessellation Inverse operator for VariatIonal Data assimilation (VIVID), that incorporates a DL inverse operator into the assimilation objective function. By leveraging the capabilities of the Voronoi-tessellation and convolutional neural networks, VIVID is adept at handling sparse, unstructured, and time-varying sensor data. Furthermore, the incorporation of the DL inverse operator establishes a direct link between observation and state space, leading to a reduction in the number of minimization steps required for DA. Additionally, VIVID can be seamlessly integrated with Proper Orthogonal Decomposition (POD) to develop an end-to-end reduced-order DA scheme, which can further expedite field reconstruction. Numerical experiments in a fluid dynamics system demonstrate that VIVID can significantly outperform existing DA and DL algorithms. The robustness of VIVID is also accessed through the application of various levels of prior error, the utilization of varying numbers of sensors, and the misspecification of error covariance in DA.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 20 of its 56 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

20 of 56 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  3. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  4. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  5. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 75
  6. 2016Neural Encoding and Decoding with Deep Learning for Dynamic Natural Visioncited by 60
  7. 2019A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problemscited by 59
  8. 2021Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learningcited by 57
  9. 2020Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernelscited by 53
  10. 2022GNN-Surrogate: A Hierarchical and Adaptive Graph Neural Network for Parameter Space Exploration of Unstructured-Mesh Ocean Simulationscited by 32
  11. 2021Latent Space Data Assimilation by using Deep Learningcited by 30
  12. 2021Variational Data Assimilation with a Learned Inverse Observation Operatorcited by 28
8 more
  1. 2020Supervised learning from noisy observations: Combining machine-learning techniques with data assimilationcited by 27
  2. 2022Generalised Latent Assimilation in Heterogeneous Reduced Spaces with Machine Learning Surrogate Modelscited by 26
  3. 2018Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalizationcited by 21
  4. 2021A comparison of combined data assimilation and machine learning methods for offline and online model error correctioncited by 13
  5. 2022Multimodal 4DVarNets for the reconstruction of sea surface dynamics from SST-SSH synergiescited by 13
  6. 2022Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systemscited by 12
  7. 2021Observation Error Covariance Specification in Dynamical Systems for Data assimilation using Recurrent Neural Networkscited by 7
  8. 2013Low-rank Approximations for Computing Observation Impact in 4D-Var Data Assimilationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024Spatially-Aware Diffusion Models with Cross-Attention for Global Field Reconstruction with Sparse Observationscited by 13
  2. 2025Machine learning for modelling unstructured grid data in computational physics: a reviewcited by 12
  3. 2024Deep Operator Learning for High-Fidelity Fluid Flow Field Reconstruction from Sparse Sensor Measurementscited by 9
  4. 2025From Models To Experiments: Shallow Recurrent Decoder Networks on the DYNASTY Experimental Facilitycited by 6
  5. 2024Dynamical system prediction from sparse observations using deep neural networks with Voronoi tessellation and physics constraintcited by 5
  6. 2025Interpretable Diagnostics and Adaptive Data Assimilation for Neural ODEs via Discrete Empirical Interpolationcited by 1
  7. 2026Rooftop Wind Field Reconstruction Using Sparse Sensors: From Deterministic to Generative Learning Methodscited by 1
  8. 2026Neural Operator Processes for Probabilistic Operator Learning under Partial Observationscited by 1
  9. 2025Learning Mappings in Mesh-based Simulationscited by 0
  10. 2025Learning to Reconstruct Temperature Field from Sparse Observations with Implicit Physics Priorscited by 0
  11. 2026Latent attention on masked patches for flow reconstructioncited by 0
  12. 2026Spatiotemporal System Forecasting with Irregular Time Steps via Masked Autoencodercited by 0
2 more
  1. 2026GLU: Global-Local-Uncertainty Fusion for Scalable Spatiotemporal Reconstruction and Forecastingcited by 0
  2. 2026Variational Parameter Calibration with Physics-Aware Latent-Space Surrogatescited by 0