# Dimensionality Reduction for Binary Data through the Projection of Natural Parameters

arXiv: https://arxiv.org/abs/1510.06112 · HTML: https://mlsanity.com/paper/1510.06112/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2015.

## Built on by (most cited first, 6 of 6)

- [Spherical Rotation Dimension Reduction with Geometric Loss Functions](https://mlsanity.com/paper/2204.10975/) (2022)
- [A majorization-minimization algorithm for nonnegative binary matrix factorization](https://mlsanity.com/paper/2204.09741/) (2022)
- [Layerwise Bregman Representation Learning with Applications to Knowledge Distillation](https://mlsanity.com/paper/2209.07080/) (2022)
- [Maximum a Posteriori Inference of Random Dot Product Graphs via Conic Programming](https://mlsanity.com/paper/2101.02180/) (2021)
- [Assessing putative bias in prediction of anti-microbial resistance from real-world genotyping data under explicit causal assumptions](https://mlsanity.com/paper/2107.03383/) (2021)
- [Local Linear Convergence of Gradient Methods for Subspace Optimization via Strict Complementarity](https://mlsanity.com/paper/2202.04020/) (2022)

## Abstract (arXiv)

Principal component analysis (PCA) for binary data, known as logistic PCA, has become a popular alternative to dimensionality reduction of binary data. It is motivated as an extension of ordinary PCA by means of a matrix factorization, akin to the singular value decomposition, that maximizes the Bernoulli log-likelihood. We propose a new formulation of logistic PCA which extends Pearson's formulation of a low dimensional data representation with minimum error to binary data. Our formulation does not require a matrix factorization, as previous methods do, but instead looks for projections of the natural parameters from the saturated model. Due to this difference, the number of parameters does not grow with the number of observations and the principal component scores on new data can be computed with simple matrix multiplication. We derive explicit solutions for data matrices of special structure and provide computationally efficient algorithms for solving for the principal component loadings. Through simulation experiments and an analysis of medical diagnoses data, we compare our formulation of logistic PCA to the previous formulation as well as ordinary PCA to demonstrate its benefits.