# Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks

arXiv: https://arxiv.org/abs/1808.03620 · HTML: https://mlsanity.com/paper/1808.03620/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2018.

## Built on by (most cited first, 30 of 37)

- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems](https://mlsanity.com/paper/2409.20175/) (2024)
- [Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers](https://mlsanity.com/paper/2103.09360/) (2021)
- [Bridging observation, theory and numerical simulation of the ocean using Machine Learning](https://mlsanity.com/paper/2104.12506/) (2021)
- [Neural Dynamical Operator: Continuous Spatial-Temporal Model with Gradient-Based and Derivative-Free Optimization Methods](https://mlsanity.com/paper/2311.11798/) (2023)
- [Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networks](https://mlsanity.com/paper/2101.03906/) (2021)
- [Gradient-free training of neural ODEs for system identification and control using ensemble Kalman inversion](https://mlsanity.com/paper/2307.07882/) (2023)
- [Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods](https://mlsanity.com/paper/2104.03384/) (2021)
- [Non-Asymptotic Analysis of Ensemble Kalman Updates: Effective Dimension and Localization](https://mlsanity.com/paper/2208.03246/) (2022)
- [Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features](https://mlsanity.com/paper/2407.00584/) (2024)
- [Second Order Ensemble Langevin Method for Sampling and Inverse Problems](https://mlsanity.com/paper/2208.04506/) (2022)
- [Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion](https://mlsanity.com/paper/2303.07392/) (2023)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Active Learning of Model Discrepancy with Bayesian Experimental Design](https://mlsanity.com/paper/2502.05372/) (2025)
- [Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priors](https://mlsanity.com/paper/2510.10968/) (2025)
- [Online Calibration of Deep Learning Sub-Models for Hybrid Numerical Modeling Systems](https://mlsanity.com/paper/2311.10665/) (2023)
- [Uncertainty quantification for deeponets with ensemble kalman inversion](https://mlsanity.com/paper/2403.03444/) (2024)
- [Flow Annealed Kalman Inversion for Gradient-Free Inference in Bayesian Inverse Problems](https://mlsanity.com/paper/2309.11490/) (2023)
- [Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence](https://mlsanity.com/paper/2312.00907/) (2023)
- [Derivative-Free Diffusion Manifold-Constrained Gradient for Unified XAI](https://mlsanity.com/paper/2411.15265/) (2024)
- [Scalable Bayesian Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.08501/) (2025)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [A unified method of data assimilation and turbulence modeling for separated flows at high Reynolds numbers](https://mlsanity.com/paper/2211.00601/) (2022)
- [Private Knowledge Sharing in Distributed Learning: A Survey](https://mlsanity.com/paper/2402.06682/) (2024)
- [Adjoint-based online learning of two-layer quasi-geostrophic baroclinic turbulence](https://mlsanity.com/paper/2411.14106/) (2024)
- [Nesterov Acceleration for Ensemble Kalman Inversion and Variants](https://mlsanity.com/paper/2501.08779/) (2025)
- [Variational Kalman Filtering with Hinf-Based Correction for Robust Bayesian Learning in High Dimensions](https://mlsanity.com/paper/2204.13089/) (2022)
- [Reweighted Interacting Langevin Diffusions: an Accelerated Sampling Methodfor Optimization](https://mlsanity.com/paper/2301.12640/) (2023)
- [Affine Invariant Ensemble Transform Methods to Improve Predictive Uncertainty in Neural Networks](https://mlsanity.com/paper/2309.04742/) (2023)
- [Training neural networks without backpropagation using particles](https://mlsanity.com/paper/2412.05667/) (2024)

## Abstract (arXiv)

The standard probabilistic perspective on machine learning gives rise to empirical risk-minimization tasks that are frequently solved by stochastic gradient descent (SGD) and variants thereof. We present a formulation of these tasks as classical inverse or filtering problems and, furthermore, we propose an efficient, gradient-free algorithm for finding a solution to these problems using ensemble Kalman inversion (EKI). Applications of our approach include offline and online supervised learning with deep neural networks, as well as graph-based semi-supervised learning. The essence of the EKI procedure is an ensemble based approximate gradient descent in which derivatives are replaced by differences from within the ensemble. We suggest several modifications to the basic method, derived from empirically successful heuristics developed in the context of SGD. Numerical results demonstrate wide applicability and robustness of the proposed algorithm.