mlsanityData to 14 Oct 2026

An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systems

arXiv 2309.00983 ↗First posted 2 Sep 2023stat.ML cs.LG math.OCcited by 38

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

We propose an ensemble score filter (EnSF) for solving high-dimensional nonlinear filtering problems with superior accuracy. A major drawback of existing filtering methods, e.g., particle filters or ensemble Kalman filters, is the low accuracy in handling high-dimensional and highly nonlinear problems. EnSF attacks this challenge by exploiting the score-based diffusion model, defined in a pseudo-temporal domain, to characterizing the evolution of the filtering density. EnSF stores the information of the recursively updated filtering density function in the score function, instead of storing the information in a set of finite Monte Carlo samples (used in particle filters and ensemble Kalman filters). Unlike existing diffusion models that train neural networks to approximate the score function, we develop a training-free score estimation that uses a mini-batch-based Monte Carlo estimator to directly approximate the score function at any pseudo-spatial-temporal location, which provides sufficient accuracy in solving high-dimensional nonlinear problems as well as saves a tremendous amount of time spent on training neural networks. High-dimensional Lorenz-96 systems are used to demonstrate the performance of our method. EnSF provides surprising performance, compared with the state-of-the-art Local Ensemble Transform Kalman Filter method, in reliably and efficiently tracking extremely high-dimensional Lorenz systems (up to 1,000,000 dimensions) with highly nonlinear observation processes.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 24 of its 60 references that are in the map. First posted on arXiv 2 Sep 2023.

Builds on

24 of 60 references.

References · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  9. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  10. 2021Variational Diffusion Modelscited by 1,153
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
12 more
  1. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  2. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  3. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  4. 2021Deblurring via Stochastic Refinementcited by 249
  5. 2020Learning Gradient Fields for Shape Generationcited by 245
  6. 2022Diffusion models as plug-and-play priorscited by 231
  7. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  8. 2021Score-Based Point Cloud Denoisingcited by 83
  9. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  10. 2019Coupling techniques for nonlinear ensemble filteringcited by 48
  11. 2021Stochastic Image Denoising by Sampling from the Posterior Distributioncited by 45
  12. 2015On dimension reduction in Gaussian filterscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 11; 2026: 22.

38 papers.

Sort:

By area:

  1. 2024Latent-EnSF: A Latent Ensemble Score Filter for High-Dimensional Data Assimilation with Sparse Observation Datacited by 22
  2. 2024A Training-Free Conditional Diffusion Model for Learning Stochastic Dynamical Systemscited by 19
  3. 2025FlowDAS: A Stochastic Interpolant-based Framework for Data Assimilationcited by 18
  4. 2025Flow Matching for Efficient and Scalable Data Assimilationcited by 12
  5. 2024LD-EnSF: Synergizing Latent Dynamics with Ensemble Score Filters for Fast Data Assimilation with Sparse Observationscited by 11
  6. 2024State-observation augmented diffusion model for nonlinear assimilation with unknown dynamicscited by 8
  7. 2025Bifidelity Parameter Estimation Using Conditional Diffusion Modelscited by 7
  8. 2024A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamicscited by 5
  9. 2025DAISI: Data Assimilation with Inverse Sampling using Stochastic Interpolantscited by 5
  10. 2026LEVDA: Latent Ensemble Variational Data Assimilation via Differentiable Dynamicscited by 4
  11. 2026Training-Free Bayesian Filtering with Generative Emulatorscited by 4
  12. 2025Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systemscited by 3
26 more
  1. 2025Generative AI Models for Learning Flow Maps of Stochastic Dynamical Systems in Bounded Domainscited by 3
  2. 2025The Ensemble Schr"odinger Bridge filter for Nonlinear Data Assimilationcited by 3
  3. 2026Latent Autoencoder Ensemble Kalman Filter for Nonlinear Data assimilationcited by 3
  4. 2026A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systemscited by 3
  5. 2026Closed-form conditional diffusion models for data assimilationcited by 3
  6. 2025Diffusion differentiable resamplingcited by 2
  7. 2026Amortized Filtering and Smoothing with Conditional Normalizing Flowscited by 2
  8. 2026Global Attention with Linear Complexity for Exascale Generative Data Assimilation in Earth System Predictioncited by 2
  9. 2026Pathwise Learning of Stochastic Dynamical Systems with Partial Observationscited by 1
  10. 2026Rethinking Forward Processes for Score-Based Nonlinear Data Assimilation in High Dimensionscited by 1
  11. 2026Toward AI-Driven Digital Twins for Metropolitan Floods: A Conditional Latent Dynamics Network Surrogate of the Shallow Water Equationscited by 1
  12. 2025State Estimation Using Particle Filtering in Adaptive Machine Learning Methods: Integrating Q-Learning and NEAT Algorithms with Noisy Radar Measurementscited by 0
  13. 2025PnP-DA: Towards Principled Plug-and-Play Integration of Variational Data Assimilation and Generative Modelscited by 0
  14. 2025Fire-EnSF: Wildfire Spread Data Assimilation using Ensemble Score Filtercited by 0
  15. 2026Gradual Fine-Tuning for Flow Matching Modelscited by 0
  16. 2026Diffusion Model-Based Data Assimilation for Real-World Energy Consumption Forecastingcited by 0
  17. 2026Generative Model Proposal based Particle Filtering for Data Assimilationcited by 0
  18. 2026Ensemble Controlled-Flow Filtering for Implicit Data Assimilationcited by 0
  19. 2026SDE Guided Monte Carlo Reinforcement Learning: A Stochastic Maximum Principle Approach for Robust Decision Making in Noisy Environmentscited by 0
  20. 2026Generative models for simulation based filtering: Formulations and Empirical Comparisonscited by 0
  21. 2026FREESIA: Covariance-Aware Posterior Transport for Expressive and Scalable Data Assimilationcited by 0
  22. 2026Data-Assimilation-Assisted Reinforcement Learning for Power Grid Control under Load Uncertaintycited by 0
  23. 2026AECSF: Adaptive Ensemble Conditional Score Filtering for High-Dimensional Nonlinear Data Assimilationcited by 0
  24. 2026DAWIS: Data Assimilation with Windowed Inverse Sampling via Multitask Interpolantscited by 0
  25. 2026A Unified Framework for Bayesian Data Assimilation with Generative Models and Observation Interpolantscited by 0
  26. 2026LD-EnFF: Latent-Dynamics Ensemble Flow Filtering for Data Assimilation with Sparse Observationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025An Efficient Conditional Score-based Filter for High Dimensional Nonlinear Filtering Problems
  2. 2024A competitive baseline for deep learning enhanced data assimilation using conditional Gaussian ensemble Kalman filtering
  3. 2025Ensemble score filter with image inpainting for data assimilation in tracking surface quasi-geostrophic dynamics with partial observations
  4. 2025TrackDiffuser: Nearly Model-Free Bayesian Filtering with Diffusion Model
  5. 2026EnJoi: Ensemble Joint Score Filter for Generative Data Assimilation
  6. 2024Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning
  7. 2026SURGE: Approximation and Training Free Particle Filter for Diffusion Surrogate
  8. 2025Adversarial Transform Particle Filters