# Towards Comprehensive Information-theoretic Multi-view Learning

arXiv: https://arxiv.org/abs/2509.02084 · HTML: https://mlsanity.com/paper/2509.02084/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 55 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [How Does Information Bottleneck Help Deep Learning?](https://mlsanity.com/paper/2305.18887/) (2023)
- [Multi-view Contrastive Graph Clustering](https://mlsanity.com/paper/2110.11842/) (2021)
- [Factorized Contrastive Learning: Going Beyond Multi-view Redundancy](https://mlsanity.com/paper/2306.05268/) (2023)
- [Variational Recurrent Auto-Encoders](https://mlsanity.com/paper/1412.6581/) (2014)
- [Reliable Conflictive Multi-View Learning](https://mlsanity.com/paper/2402.16897/) (2024)
- [Uncertainty-Aware Multi-View Representation Learning](https://mlsanity.com/paper/2201.05776/) (2022)
- [On Unifying Multi-View Self-Representations for Clustering by Tensor Multi-Rank Minimization](https://mlsanity.com/paper/1610.07126/) (2016)
- [Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering](https://mlsanity.com/paper/2204.12496/) (2022)
- [A Novel Approach for Effective Multi-View Clustering with Information-Theoretic Perspective](https://mlsanity.com/paper/2309.13989/) (2023)
- [Gacs-Korner Common Information Variational Autoencoder](https://mlsanity.com/paper/2205.12239/) (2022)
- [Reliable Representation Learning for Incomplete Multi-View Missing Multi-Label Classification](https://mlsanity.com/paper/2303.17117/) (2023)
- [Enhanced Latent Multi-view Subspace Clustering](https://mlsanity.com/paper/2312.14763/) (2023)
- [Differentiable Information Bottleneck for Deterministic Multi-view Clustering](https://mlsanity.com/paper/2403.15681/) (2024)
- [Discovering Common Information in Multi-view Data](https://mlsanity.com/paper/2406.15043/) (2024)
- [Generalized Trusted Multi-view Classification Framework with Hierarchical Opinion Aggregation](https://mlsanity.com/paper/2411.03713/) (2024)

## Built on by (most cited first, 1 of 1)

- [IndiSeek learns information-guided disentangled representations](https://mlsanity.com/paper/2509.21584/) (2025)

## Abstract (arXiv)

Information theory has inspired numerous advancements in multi-view learning. Most multi-view methods incorporating information-theoretic principles rely an assumption called multi-view redundancy which states that common information between views is necessary and sufficient for down-stream tasks. This assumption emphasizes the importance of common information for prediction, but inherently ignores the potential of unique information in each view that could be predictive to the task. In this paper, we propose a comprehensive information-theoretic multi-view learning framework named CIML, which discards the assumption of multi-view redundancy. Specifically, CIML considers the potential predictive capabilities of both common and unique information based on information theory. First, the common representation learning maximizes Gacs-Korner common information to extract shared features and then compresses this information to learn task-relevant representations based on the Information Bottleneck (IB). For unique representation learning, IB is employed to achieve the most compressed unique representation for each view while simultaneously minimizing the mutual information between unique and common representations, as well as among different unique representations. Importantly, we theoretically prove that the learned joint representation is predictively sufficient for the downstream task. Extensive experimental results have demonstrated the superiority of our model over several state-of-art methods. The code is released on CIML.