# Exact nonlinear state estimation

arXiv: https://arxiv.org/abs/2310.10976 · HTML: https://mlsanity.com/paper/2310.10976/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Coupling techniques for nonlinear ensemble filtering](https://mlsanity.com/paper/1907.00389/) (2019)
- [An overview of differentiable particle filters for data-adaptive sequential Bayesian inference](https://mlsanity.com/paper/2302.09639/) (2023)
- [Ensemble Kalman Methods: A Mean Field Perspective](https://mlsanity.com/paper/2209.11371/) (2022)
- [Differentiable Particle Filters through Conditional Normalizing Flow](https://mlsanity.com/paper/2107.00488/) (2021)
- [An Optimal Transport Formulation of Bayes' Law for Nonlinear Filtering Algorithms](https://mlsanity.com/paper/2203.11869/) (2022)
- [A comparison of nonlinear extensions to the ensemble Kalman filter: Gaussian Anamorphosis and Two-Step Ensemble Filters](https://mlsanity.com/paper/2107.07475/) (2021)
- [Sampling-Free Probabilistic Deep State-Space Models](https://mlsanity.com/paper/2309.08256/) (2023)

## Built on by (most cited first, 4 of 4)

- [Learning Enhanced Ensemble Filters](https://mlsanity.com/paper/2504.17836/) (2025)
- [A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics](https://mlsanity.com/paper/2407.12168/) (2024)
- [Closed-form conditional diffusion models for data assimilation](https://mlsanity.com/paper/2603.21291/) (2026)
- [Preserving linear invariants in ensemble filtering methods](https://mlsanity.com/paper/2404.14328/) (2024)

## Abstract (arXiv)

The majority of data assimilation (DA) methods in the geosciences are based on Gaussian assumptions. While these assumptions facilitate efficient algorithms, they cause analysis biases and subsequent forecast degradations. Non-parametric, particle-based DA algorithms have superior accuracy, but their application to high-dimensional models still poses operational challenges. Drawing inspiration from recent advances in the field of generative artificial intelligence (AI), this article introduces a new nonlinear estimation theory which attempts to bridge the existing gap in DA methodology. Specifically, a Conjugate Transform Filter (CTF) is derived and shown to generalize the celebrated Kalman filter to arbitrarily non-Gaussian distributions. The new filter has several desirable properties, such as its ability to preserve statistical relationships in the prior state and convergence to highly accurate observations. An ensemble approximation of the new theory (ECTF) is also presented and validated using idealized statistical experiments that feature bounded quantities with non-Gaussian distributions, a prevalent challenge in Earth system models. Results from these experiments indicate that the greatest benefits from ECTF occur when observation errors are small relative to the forecast uncertainty and when state variables exhibit strong nonlinear dependencies. Ultimately, the new filtering theory offers exciting avenues for improving conventional DA algorithms through their principled integration with AI techniques.