# Information Projection and Approximate Inference for Structured Sparse Variables

arXiv: https://arxiv.org/abs/1607.03204 · HTML: https://mlsanity.com/paper/1607.03204/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2016.

## Built on by (most cited first, 1 of 1)

- [DP-HyPO: An Adaptive Private Hyperparameter Optimization Framework](https://mlsanity.com/paper/2306.05734/) (2023)

## Abstract (arXiv)

Approximate inference via information projection has been recently introduced as a general-purpose approach for efficient probabilistic inference given sparse variables. This manuscript goes beyond classical sparsity by proposing efficient algorithms for approximate inference via information projection that are applicable to any structure on the set of variables that admits enumeration using a matroid. We show that the resulting information projection can be reduced to combinatorial submodular optimization subject to matroid constraints. Further, leveraging recent advances in submodular optimization, we provide an efficient greedy algorithm with strong optimization-theoretic guarantees. The class of probabilistic models that can be expressed in this way is quite broad and, as we show, includes group sparse regression, group sparse principal components analysis and sparse canonical correlation analysis, among others. Moreover, empirical results on simulated data and high dimensional neuroimaging data highlight the superior performance of the information projection approach as compared to established baselines for a range of probabilistic models.