# Compressive Independent Component Analysis: Theory and Algorithms

arXiv: https://arxiv.org/abs/2110.08045 · HTML: https://mlsanity.com/paper/2110.08045/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 60 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 6 of 6)

- [Turning Big data into tiny data: Constant-size coresets for k-means, PCA and projective clustering](https://mlsanity.com/paper/1807.04518/) (2018)
- [Practical sketching algorithms for low-rank matrix approximation](https://mlsanity.com/paper/1609.00048/) (2016)
- [Sketching for Large-Scale Learning of Mixture Models](https://mlsanity.com/paper/1606.02838/) (2016)
- [Compressive Statistical Learning with Random Feature Moments](https://mlsanity.com/paper/1706.07180/) (2017)
- [Compressive Learning for Semi-Parametric Models](https://mlsanity.com/paper/1910.10024/) (2019)
- [Randomized Independent Component Analysis](https://mlsanity.com/paper/1609.06942/) (2016)

## Abstract (arXiv)

Compressive learning forms the exciting intersection between compressed sensing and statistical learning where one exploits forms of sparsity and structure to reduce the memory and/or computational complexity of the learning task. In this paper, we look at the independent component analysis (ICA) model through the compressive learning lens. In particular, we show that solutions to the cumulant based ICA model have particular structure that induces a low dimensional model set that resides in the cumulant tensor space. By showing a restricted isometry property holds for random cumulants e.g. Gaussian ensembles, we prove the existence of a compressive ICA scheme. Thereafter, we propose two algorithms of the form of an iterative projection gradient (IPG) and an alternating steepest descent (ASD) algorithm for compressive ICA, where the order of compression asserted from the restricted isometry property is realised through empirical results. We provide analysis of the CICA algorithms including the effects of finite samples. The effects of compression are characterised by a trade-off between the sketch size and the statistical efficiency of the ICA estimates. By considering synthetic and real datasets, we show the substantial memory gains achieved over well-known ICA algorithms by using one of the proposed CICA algorithms. Finally, we conclude the paper with open problems including interesting challenges from the emerging field of compressive learning.