# Efficient Alternating Minimization Solvers for Wyner Multi-View Unsupervised Learning

arXiv: https://arxiv.org/abs/2303.15866 · HTML: https://mlsanity.com/paper/2303.15866/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems](https://mlsanity.com/paper/1706.06461/) (2017)
- [Calculus of the exponent of Kurdyka-Lojasiewicz inequality and its applications to linear convergence of first-order methods](https://mlsanity.com/paper/1602.02915/) (2016)
- [Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence results](https://mlsanity.com/paper/1709.05747/) (2017)
- [Optimal rates of entropy estimation over Lipschitz balls](https://mlsanity.com/paper/1711.02141/) (2017)
- [Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems](https://mlsanity.com/paper/1409.8444/) (2014)
- [Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding](https://mlsanity.com/paper/1905.11741/) (2019)
- [Nonconvex Lagrangian-Based Optimization: Monitoring Schemes and Global Convergence](https://mlsanity.com/paper/1801.03013/) (2018)
- [A Linearly Convergent Douglas-Rachford Splitting Solver for Markovian Information-Theoretic Optimization Problems](https://mlsanity.com/paper/2203.07527/) (2022)
- [Common Information Components Analysis](https://mlsanity.com/paper/2002.00779/) (2020)
- [A Provably Convergent Information Bottleneck Solution via ADMM](https://mlsanity.com/paper/2102.04729/) (2021)

## Built on by (most cited first, 1 of 1)

- [An Efficient Difference-of-Convex Solver for Privacy Funnel](https://mlsanity.com/paper/2403.04778/) (2024)

## Abstract (arXiv)

In this work, we adopt Wyner common information framework for unsupervised multi-view representation learning. Within this framework, we propose two novel formulations that enable the development of computational efficient solvers based on the alternating minimization principle. The first formulation, referred to as the em variational form, enjoys a linearly growing complexity with the number of views and is based on a variational-inference tight surrogate bound coupled with a Lagrangian optimization objective function. The second formulation, i.e., the em representational form, is shown to include known results as special cases. Here, we develop a tailored version from the alternating direction method of multipliers (ADMM) algorithm for solving the resulting non-convex optimization problem. In the two cases, the convergence of the proposed solvers is established in certain relevant regimes. Furthermore, our empirical results demonstrate the effectiveness of the proposed methods as compared with the state-of-the-art solvers. In a nutshell, the proposed solvers offer computational efficiency, theoretical convergence guarantees (local minima), scalable complexity with the number of views, and exceptional accuracy as compared with the state-of-the-art techniques. Our focus here is devoted to the discrete case and our results for continuous distributions are reported elsewhere.