# Discovering Common Information in Multi-view Data

arXiv: https://arxiv.org/abs/2406.15043 · HTML: https://mlsanity.com/paper/2406.15043/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [A Survey on Multi-view Learning](https://mlsanity.com/paper/1304.5634/) (2013)
- [Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering](https://mlsanity.com/paper/2106.11232/) (2021)
- [A Variational Information Bottleneck Approach to Multi-Omics Data Integration](https://mlsanity.com/paper/2102.03014/) (2021)
- [Angle-Based Joint and Individual Variation Explained](https://mlsanity.com/paper/1704.02060/) (2017)
- [Uncertainty-Aware Multi-View Representation Learning](https://mlsanity.com/paper/2201.05776/) (2022)
- [Measuring Dependence with Matrix-based Entropy Functional](https://mlsanity.com/paper/2101.10160/) (2021)
- [Deep Co-Attention Network for Multi-View Subspace Learning](https://mlsanity.com/paper/2102.07751/) (2021)
- [Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data](https://mlsanity.com/paper/1508.07416/) (2015)
- [Seeking Commonness and Inconsistencies: A Jointly Smoothed Approach to Multi-view Subspace Clustering](https://mlsanity.com/paper/2203.08060/) (2022)
- [Gacs-Korner Common Information Variational Autoencoder](https://mlsanity.com/paper/2205.12239/) (2022)
- [Optimal Randomized Approximations for Matrix based Renyi's Entropy](https://mlsanity.com/paper/2205.07426/) (2022)
- [Multi-view Information Bottleneck Without Variational Approximation](https://mlsanity.com/paper/2204.10530/) (2022)
- [D-GCCA: Decomposition-based Generalized Canonical Correlation Analysis for Multi-view High-dimensional Data](https://mlsanity.com/paper/2001.02856/) (2020)

## Built on by (most cited first, 3 of 3)

- [ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGS](https://mlsanity.com/paper/2505.23734/) (2025)
- [Towards Comprehensive Information-theoretic Multi-view Learning](https://mlsanity.com/paper/2509.02084/) (2025)
- [SAMatcher: Dense Co-Visibility Modeling via Cross-View Fusion for Scale-Imbalance Image Matching](https://mlsanity.com/paper/2606.03406/) (2026)

## Abstract (arXiv)

We introduce an innovative and mathematically rigorous definition for computing common information from multi-view data, drawing inspiration from G'acs-K"orner common information in information theory. Leveraging this definition, we develop a novel supervised multi-view learning framework to capture both common and unique information. By explicitly minimizing a total correlation term, the extracted common information and the unique information from each view are forced to be independent of each other, which, in turn, theoretically guarantees the effectiveness of our framework. To estimate information-theoretic quantities, our framework employs matrix-based R'enyi's α-order entropy functional, which forgoes the need for variational approximation and distributional estimation in high-dimensional space. Theoretical proof is provided that our framework can faithfully discover both common and unique information from multi-view data. Experiments on synthetic and seven benchmark real-world datasets demonstrate the superior performance of our proposed framework over state-of-the-art approaches.