# Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspective

arXiv: https://arxiv.org/abs/2106.07115 · HTML: https://mlsanity.com/paper/2106.07115/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 66 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Understanding self-supervised Learning Dynamics without Contrastive Pairs](https://mlsanity.com/paper/2102.06810/) (2021)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [Contrastive learning, multi-view redundancy, and linear models](https://mlsanity.com/paper/2008.10150/) (2020)
- [The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA](https://mlsanity.com/paper/1905.06642/) (2019)
- [Deep Variational Canonical Correlation Analysis](https://mlsanity.com/paper/1610.03454/) (2016)
- [Deep Generalized Canonical Correlation Analysis](https://mlsanity.com/paper/1702.02519/) (2017)
- [Information Constraints on Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1805.08672/) (2018)
- [Reliable Detection of Unknown Cell-Edge Users Via Canonical Correlation Analysis](https://mlsanity.com/paper/2003.02255/) (2020)

## Built on by (most cited first, 30 of 37)

- [Multi-View Causal Representation Learning with Partial Observability](https://mlsanity.com/paper/2311.04056/) (2023)
- [Identifiability Results for Multimodal Contrastive Learning](https://mlsanity.com/paper/2303.09166/) (2023)
- [Causal Component Analysis](https://mlsanity.com/paper/2305.17225/) (2023)
- [Understanding Masked Autoencoders via Hierarchical Latent Variable Models](https://mlsanity.com/paper/2306.04898/) (2023)
- [Causal Representation Learning Made Identifiable by Grouping of Observational Variables](https://mlsanity.com/paper/2310.15709/) (2023)
- [Identification of Nonlinear Latent Hierarchical Models](https://mlsanity.com/paper/2306.07916/) (2023)
- [Marrying Causal Representation Learning with Dynamical Systems for Science](https://mlsanity.com/paper/2405.13888/) (2024)
- [Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations](https://mlsanity.com/paper/2311.08815/) (2023)
- [Upper Bounding Barlow Twins: A Novel Filter for Multi-Relational Clustering](https://mlsanity.com/paper/2312.14066/) (2023)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [On the Value of Cross-Modal Misalignment in Multimodal Representation Learning](https://mlsanity.com/paper/2504.10143/) (2025)
- [Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research](https://mlsanity.com/paper/2504.13101/) (2025)
- [On Finite-Sample Identifiability of Contrastive Learning-Based Nonlinear Independent Component Analysis](https://mlsanity.com/paper/2206.06593/) (2022)
- [Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck](https://mlsanity.com/paper/2309.07200/) (2023)
- [Revisiting Deep Generalized Canonical Correlation Analysis](https://mlsanity.com/paper/2312.13455/) (2023)
- [Latent Processes Identification From Multi-View Time Series](https://mlsanity.com/paper/2305.08164/) (2023)
- [Towards Identifiable Unsupervised Domain Translation: A Diversified Distribution Matching Approach](https://mlsanity.com/paper/2401.09671/) (2024)
- [Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectives](https://mlsanity.com/paper/2309.00380/) (2023)
- [Disentanglement of Variations with Multimodal Generative Modeling](https://mlsanity.com/paper/2509.23548/) (2025)
- [Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability](https://mlsanity.com/paper/2510.17040/) (2025)
- [Identifiable Shared Component Analysis of Unpaired Multimodal Mixtures](https://mlsanity.com/paper/2409.19422/) (2024)
- [Provable Subspace Identification Under Post-Nonlinear Mixtures](https://mlsanity.com/paper/2210.07532/) (2022)
- [Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions](https://mlsanity.com/paper/2411.03755/) (2024)
- [Content-Style Identification via Differential Independence](https://mlsanity.com/paper/2605.17827/) (2026)
- [Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes](https://mlsanity.com/paper/2209.02876/) (2022)
- [Unsupervised discovery of the shared and private geometry in multi-view data](https://mlsanity.com/paper/2408.12091/) (2024)
- [Provable Affine Identifiability of Nonlinear CCA under Latent Distributional Priors](https://mlsanity.com/paper/2510.04758/) (2025)
- [Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability](https://mlsanity.com/paper/2603.21510/) (2026)
- [Communication-Efficient Federated Linear and Deep Generalized Canonical Correlation Analysis](https://mlsanity.com/paper/2109.12400/) (2021)
- [Identifiability of a statistical model with two latent vectors: Importance of the dimensionality relation and application to graph embedding](https://mlsanity.com/paper/2405.19760/) (2024)

## Abstract (arXiv)

Multiple views of data, both naturally acquired (e.g., image and audio) and artificially produced (e.g., via adding different noise to data samples), have proven useful in enhancing representation learning. Natural views are often handled by multiview analysis tools, e.g., (deep) canonical correlation analysis [(D)CCA], while the artificial ones are frequently used in self-supervised learning (SSL) paradigms, e.g., BYOL and Barlow Twins. Both types of approaches often involve learning neural feature extractors such that the embeddings of data exhibit high cross-view correlations. Although intuitive, the effectiveness of correlation-based neural embedding is mostly empirically validated. This work aims to understand latent correlation maximization-based deep multiview learning from a latent component identification viewpoint. An intuitive generative model of multiview data is adopted, where the views are different nonlinear mixtures of shared and private components. Since the shared components are view/distortion-invariant, representing the data using such components is believed to reveal the identity of the samples effectively and robustly. Under this model, latent correlation maximization is shown to guarantee the extraction of the shared components across views (up to certain ambiguities). In addition, it is further shown that the private information in each view can be provably disentangled from the shared using proper regularization design. A finite sample analysis, which has been rare in nonlinear mixture identifiability study, is also presented. The theoretical results and newly designed regularization are tested on a series of tasks.