# Maximum likelihood inference for high-dimensional problems with multiaffine variable relations

arXiv: https://arxiv.org/abs/2409.03495 · HTML: https://mlsanity.com/paper/2409.03495/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 53 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Gradientless Descent: High-Dimensional Zeroth-Order Optimization](https://mlsanity.com/paper/1911.06317/) (2019)
- [Bayesian Tensor Regression](https://mlsanity.com/paper/1509.06490/) (2015)
- [Generalized Kalman Smoothing: Modeling and Algorithms](https://mlsanity.com/paper/1609.06369/) (2016)
- [Learning Discrete Bayesian Networks from Continuous Data](https://mlsanity.com/paper/1512.02406/) (2015)
- [Regularization and Bayesian Learning in Dynamical Systems: Past, Present and Future](https://mlsanity.com/paper/1511.01543/) (2015)
- [Sensitivity-conditioning: Beyond Singular Perturbation for Control Design on Multiple Time Scales](https://mlsanity.com/paper/2101.04367/) (2021)
- [A Multivariate Discretization Method for Learning Bayesian Networks from Mixed Data](https://mlsanity.com/paper/1301.7403/) (2013)
- [Outlier robust system identification: a Bayesian kernel-based approach](https://mlsanity.com/paper/1312.6317/) (2013)
- [Robust online joint state/input/parameter estimation of linear systems](https://mlsanity.com/paper/2204.05663/) (2022)

## Abstract (arXiv)

Maximum Likelihood Estimation of continuous variable models can be very challenging in high dimensions, due to potentially complex probability distributions. The existence of multiple interdependencies among variables can make it very difficult to establish convergence guarantees. This leads to a wide use of brute-force methods, such as grid searching and Monte-Carlo sampling and, when applicable, complex and problem-specific algorithms. In this paper, we consider inference problems where the variables are related by multiaffine expressions. We propose a novel Alternating and Iteratively-Reweighted Least Squares (AIRLS) algorithm, and prove its convergence for problems with Generalized Normal Distributions. We also provide an efficient method to compute the variance of the estimates obtained using AIRLS. Finally, we show how the method can be applied to graphical statistical models. We perform numerical experiments on several inference problems, showing significantly better performance than state-of-the-art approaches in terms of scalability, robustness to noise, and convergence speed due to an empirically observed super-linear convergence rate.