# Bayesian Inference with Nonlinear Generative Models: Comments on Secure Learning

arXiv: https://arxiv.org/abs/2201.09986 · HTML: https://mlsanity.com/paper/2201.09986/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 96 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Statistical physics of inference: Thresholds and algorithms](https://mlsanity.com/paper/1511.02476/) (2015)
- [Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models](https://mlsanity.com/paper/1708.03395/) (2017)
- [The role of regularization in classification of high-dimensional noisy Gaussian mixture](https://mlsanity.com/paper/2002.11544/) (2020)
- [Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions](https://mlsanity.com/paper/2106.03791/) (2021)
- [Phase retrieval in high dimensions: Statistical and computational phase transitions](https://mlsanity.com/paper/2006.05228/) (2020)
- [The estimation error of general first order methods](https://mlsanity.com/paper/2002.12903/) (2020)
- [Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View](https://mlsanity.com/paper/2011.07729/) (2020)
- [Notes on computational-to-statistical gaps: predictions using statistical physics](https://mlsanity.com/paper/1803.11132/) (2018)
- [Exact asymptotics for phase retrieval and compressed sensing with random generative priors](https://mlsanity.com/paper/1912.02008/) (2019)
- [All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation](https://mlsanity.com/paper/2006.07971/) (2020)
- [Asymptotic Bayes risk for Gaussian mixture in a semi-supervised setting](https://mlsanity.com/paper/1907.03792/) (2019)
- [Statistical limits of dictionary learning: random matrix theory and the spectral replica method](https://mlsanity.com/paper/2109.06610/) (2021)
- [Optimization of Mean-field Spin Glasses](https://mlsanity.com/paper/2001.00904/) (2020)
- [Approximate Survey Propagation for Statistical Inference](https://mlsanity.com/paper/1807.01296/) (2018)
- [Performance of Bayesian linear regression in a model with mismatch](https://mlsanity.com/paper/2107.06936/) (2021)
- [Statistical Estimation: From Denoising to Sparse Regression and Hidden Cliques](https://mlsanity.com/paper/1409.5557/) (2014)
- [Optimizing Mean Field Spin Glasses with External Field](https://mlsanity.com/paper/2105.03506/) (2021)

## Abstract (arXiv)

Unlike the classical linear model, nonlinear generative models have been addressed sparsely in the literature of statistical learning. This work aims to bringing attention to these models and their secrecy potential. To this end, we invoke the replica method to derive the asymptotic normalized cross entropy in an inverse probability problem whose generative model is described by a Gaussian random field with a generic covariance function. Our derivations further demonstrate the asymptotic statistical decoupling of the Bayesian estimator and specify the decoupled setting for a given nonlinear model. The replica solution depicts that strictly nonlinear models establish an all-or-nothing phase transition: There exists a critical load at which the optimal Bayesian inference changes from perfect to an uncorrelated learning. Based on this finding, we design a new secure coding scheme which achieves the secrecy capacity of the wiretap channel. This interesting result implies that strictly nonlinear generative models are perfectly secured without any secure coding. We justify this latter statement through the analysis of an illustrative model for perfectly secure and reliable inference.