# Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Games

arXiv: https://arxiv.org/abs/2206.09120 · HTML: https://mlsanity.com/paper/2206.09120/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 68 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction](https://mlsanity.com/paper/2006.08558/) (2020)
- [Geoopt: Riemannian Optimization in PyTorch](https://mlsanity.com/paper/2005.02819/) (2020)
- [Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimization](https://mlsanity.com/paper/1807.04252/) (2018)
- [EigenGame: PCA as a Nash Equilibrium](https://mlsanity.com/paper/2010.00554/) (2020)
- [EigenGAN: Layer-Wise Eigen-Learning for GANs](https://mlsanity.com/paper/2104.12476/) (2021)
- [An Introduction to Image Synthesis with Generative Adversarial Nets](https://mlsanity.com/paper/1803.04469/) (2018)
- [Convergence of Learning Dynamics in Stackelberg Games](https://mlsanity.com/paper/1906.01217/) (2019)
- [GANs May Have No Nash Equilibria](https://mlsanity.com/paper/2002.09124/) (2020)
- [Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction](https://mlsanity.com/paper/2111.06636/) (2021)
- [Incremental Learning of Structured Memory via Closed-Loop Transcription](https://mlsanity.com/paper/2202.05411/) (2022)
- [Structured Local Optima in Sparse Blind Deconvolution](https://mlsanity.com/paper/1806.00338/) (2018)
- [Deconstructing Generative Adversarial Networks](https://mlsanity.com/paper/1901.09465/) (2019)
- [Understanding GANs: the LQG Setting](https://mlsanity.com/paper/1710.10793/) (2017)
- [Variational Clustering: Leveraging Variational Autoencoders for Image Clustering](https://mlsanity.com/paper/2005.04613/) (2020)
- [Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias](https://mlsanity.com/paper/2112.06868/) (2021)
- [Permutation methods for factor analysis and PCA](https://mlsanity.com/paper/1710.00479/) (2017)
- [Short-and-Sparse Deconvolution -- A Geometric Approach](https://mlsanity.com/paper/1908.10959/) (2019)
- [LoGAN: Generating Logos with a Generative Adversarial Neural Network Conditioned on color](https://mlsanity.com/paper/1810.10395/) (2018)
- [Complete Dictionary Learning via ℓp-norm Maximization](https://mlsanity.com/paper/2002.10043/) (2020)
- [Selecting the number of components in PCA via random signflips](https://mlsanity.com/paper/2012.02985/) (2020)
- [Deterministic parallel analysis: An improved method for selecting factors and principal components](https://mlsanity.com/paper/1711.04155/) (2017)
- [Consistency of invariance-based randomization tests](https://mlsanity.com/paper/2104.12260/) (2021)

## Built on by (most cited first, 4 of 4)

- [White-Box Transformers via Sparse Rate Reduction](https://mlsanity.com/paper/2306.01129/) (2023)
- [On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence](https://mlsanity.com/paper/2207.04630/) (2022)
- [A Global Geometric Analysis of Maximal Coding Rate Reduction](https://mlsanity.com/paper/2406.01909/) (2024)
- [Closed-Loop Transcription via Convolutional Sparse Coding](https://mlsanity.com/paper/2302.09347/) (2023)

## Abstract (arXiv)

We consider the problem of learning discriminative representations for data in a high-dimensional space with distribution supported on or around multiple low-dimensional linear subspaces. That is, we wish to compute a linear injective map of the data such that the features lie on multiple orthogonal subspaces. Instead of treating this learning problem using multiple PCAs, we cast it as a sequential game using the closed-loop transcription (CTRL) framework recently proposed for learning discriminative and generative representations for general low-dimensional submanifolds. We prove that the equilibrium solutions to the game indeed give correct representations. Our approach unifies classical methods of learning subspaces with modern deep learning practice, by showing that subspace learning problems may be provably solved using the modern toolkit of representation learning. In addition, our work provides the first theoretical justification for the CTRL framework, in the important case of linear subspaces. We support our theoretical findings with compelling empirical evidence. We also generalize the sequential game formulation to more general representation learning problems. Our code, including methods for easy reproduction of experimental results, is publically available on GitHub.