# Binary Independent Component Analysis: A Non-stationarity-based Approach

arXiv: https://arxiv.org/abs/2111.15431 · HTML: https://mlsanity.com/paper/2111.15431/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 10 of 10)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series](https://mlsanity.com/paper/2006.12107/) (2020)
- [Causal Discovery with General Non-Linear Relationships Using Non-Linear ICA](https://mlsanity.com/paper/1904.09096/) (2019)
- [Rethinking LDA: moment matching for discrete ICA](https://mlsanity.com/paper/1507.01784/) (2015)
- [Beyond CCA: Moment Matching for Multi-View Models](https://mlsanity.com/paper/1602.09013/) (2016)
- [Causal Order Identification to Address Confounding: Binary Variables](https://mlsanity.com/paper/2108.04947/) (2021)
- [Causal Discovery in a Binary Exclusive-or Skew Acyclic Model: BExSAM](https://mlsanity.com/paper/1401.5636/) (2014)

## Built on by (most cited first, 4 of 4)

- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [From superposition to sparse codes: interpretable representations in neural networks](https://mlsanity.com/paper/2503.01824/) (2025)
- [All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling](https://mlsanity.com/paper/2410.23501/) (2024)
- [On the Identifiability of Quantized Factors](https://mlsanity.com/paper/2306.16334/) (2023)

## Abstract (arXiv)

We consider independent component analysis of binary data. While fundamental in practice, this case has been much less developed than ICA for continuous data. We start by assuming a linear mixing model in a continuous-valued latent space, followed by a binary observation model. Importantly, we assume that the sources are non-stationary; this is necessary since any non-Gaussianity would essentially be destroyed by the binarization. Interestingly, the model allows for closed-form likelihood by employing the cumulative distribution function of the multivariate Gaussian distribution. In stark contrast to the continuous-valued case, we prove non-identifiability of the model with few observed variables; our empirical results imply identifiability when the number of observed variables is higher. We present a practical method for binary ICA that uses only pairwise marginals, which are faster to compute than the full multivariate likelihood. Experiments give insight into the requirements for the number of observed variables, segments, and latent sources that allow the model to be estimated.