mlsanityData to 7 Oct 2026

Local convexity of the TAP free energy and AMP convergence for Z2-synchronization

arXiv 2106.11428 ↗First posted 21 Jun 2021math.ST cs.LG math.PRcited by 15

Not on the map. Why?

Abstract (arXiv)

We study mean-field variational Bayesian inference using the TAP approach, for Z2-synchronization as a prototypical example of a high-dimensional Bayesian model. We show that for any signal strength λ > 1 (the weak-recovery threshold), there exists a unique local minimizer of the TAP free energy functional near the mean of the Bayes posterior law. Furthermore, the TAP free energy in a local neighborhood of this minimizer is strongly convex. Consequently, a natural-gradient/mirror-descent algorithm achieves linear convergence to this minimizer from a local initialization, which may be obtained by a constant number of iterates of Approximate Message Passing (AMP). This provides a rigorous foundation for variational inference in high dimensions via minimization of the TAP free energy. We also analyze the finite-sample convergence of AMP, showing that AMP is asymptotically stable at the TAP minimizer for any λ > 1, and is linearly convergent to this minimizer from a spectral initialization for sufficiently large λ. Such a guarantee is stronger than results obtainable by state evolution analyses, which only describe a fixed number of AMP iterations in the infinite-sample limit. Our proofs combine the Kac-Rice formula and Sudakov-Fernique Gaussian comparison inequality to analyze the complexity of critical points that satisfy strong convexity and stability conditions within their local neighborhoods.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 32 of its 134 references that are in the map. First posted on arXiv 21 Jun 2021.

Builds on

32 of 134 references.

References · 32

  1. 2016Variational Inference: A Review for Statisticianscited by 1,466
  2. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  3. 2016Relatively-Smooth Convex Optimization by First-Order Methods, and Applicationscited by 108
  4. 2016A Geometric Analysis of Phase Retrievalcited by 92
  5. 2016The Landscape of Empirical Risk for Non-convex Lossescited by 86
  6. 2017Concentration of tempered posteriors and of their variational approximationscited by 66
  7. 2017Frequentist Consistency of Variational Bayescited by 60
  8. 2021A unifying tutorial on Approximate Message Passingcited by 49
  9. 2017Convergence Rates of Variational Posterior Distributionscited by 45
  10. 2019Variational Bayes for high-dimensional linear regression with sparse priorscited by 44
  11. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
  12. 2020The Lasso with general Gaussian designs with applications to hypothesis testingcited by 40
20 more
  1. 2013A framework to characterize performance of LASSO algorithmscited by 38
  2. 2017The landscape of the spiked tensor modelcited by 34
  3. 2016Edward: A library for probabilistic modeling, inference, and criticismcited by 28
  4. 2016Mutual information for symmetric rank-one matrix estimation: A proof of the replica formulacited by 26
  5. 2014Spectral Clustering of Graphs with the Bethe Hessiancited by 22
  6. 2016Finite Sample Analysis of Approximate Message Passing Algorithmscited by 20
  7. 2018Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimizationcited by 20
  8. 2017Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detectioncited by 19
  9. 2021Fast Stochastic Bregman Gradient Methods: Sharp Analysis and Variance Reductioncited by 19
  10. 2020Universality of Approximate Message Passing Algorithmscited by 16
  11. 2013The Squared-Error of Generalized LASSO: A Precise Analysiscited by 15
  12. 2015Phase Transitions in Sparse PCAcited by 15
  13. 2019Landscape Complexity for the Empirical Risk of Generalized Linear Modelscited by 15
  14. 2018An Instability in Variational Inference for Topic Modelscited by 11
  15. 2021Variational Inference in high-dimensional linear regressioncited by 10
  16. 2015Non-unique games over compact groups and orientation estimation in cryo-EMcited by 7
  17. 2018The Mean-Field Approximation: Information Inequalities, Algorithms, and Complexitycited by 7
  18. 2020The Loss Surfaces of Neural Networks with General Activation Functionscited by 7
  19. 2021A spin-glass model for the loss surfaces of generative adversarial networkscited by 4
  20. 2018Perturbed Fenchel duality and first-order methodscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 6; 2024: 3; 2025: 2; 2026: 1.

15 papers.

Sort:

By area:

  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  2. 2023Learning Mixtures of Gaussians Using the DDPM Objectivecited by 56
  3. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Modelscited by 39
  4. 2023Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspectivecited by 26
  5. 2023On the Convergence of Coordinate Ascent Variational Inferencecited by 18
  6. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Modelscited by 17
  7. 2025Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formulacited by 11
  8. 2022Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energycited by 9
  9. 2023Approximate message passing from random initialization with applications to mathbbZ2 synchronizationcited by 9
  10. 2021Minimum ℓ1-norm interpolators: Precise asymptotics and multiple descentcited by 5
  11. 2023A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regressioncited by 4
  12. 2024Optimal thresholds and algorithms for a model of multi-modal learning in high dimensionscited by 4
3 more
  1. 2022The TAP free energy for high-dimensional linear regressioncited by 3
  2. 2025Dimension-Free Bounds for Generalized First-Order Methods via Gaussian Couplingcited by 2
  3. 2026Stability of Sequential and Parallel Coordinate Ascent Variational Inferencecited by 1