# Differentiable Information Bottleneck for Deterministic Multi-view Clustering

arXiv: https://arxiv.org/abs/2403.15681 · HTML: https://mlsanity.com/paper/2403.15681/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Multi-level Feature Learning for Contrastive Multi-view Clustering](https://mlsanity.com/paper/2106.11193/) (2021)
- [Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering](https://mlsanity.com/paper/2103.15069/) (2021)
- [Deep Multiview Clustering by Contrasting Cluster Assignments](https://mlsanity.com/paper/2304.10769/) (2023)
- [Deep Embedded Multi-view Clustering with Collaborative Training](https://mlsanity.com/paper/2007.13067/) (2020)
- [GCFAgg: Global and Cross-view Feature Aggregation for Multi-view Clustering](https://mlsanity.com/paper/2305.06799/) (2023)
- [Fast Multi-view Clustering via Ensembles: Towards Scalability, Superiority, and Simplicity](https://mlsanity.com/paper/2203.11572/) (2022)
- [Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering](https://mlsanity.com/paper/2204.12496/) (2022)
- [Variational Graph Generator for Multi-View Graph Clustering](https://mlsanity.com/paper/2210.07011/) (2022)
- [Deep Embedded Multi-View Clustering via Jointly Learning Latent Representations and Graphs](https://mlsanity.com/paper/2205.03803/) (2022)

## Built on by (most cited first, 3 of 3)

- [Multiview Representation Learning via Distributed Joint Latent Space Structuring](https://mlsanity.com/paper/2504.18455/) (2025)
- [Towards Comprehensive Information-theoretic Multi-view Learning](https://mlsanity.com/paper/2509.02084/) (2025)
- [SMART: Semantic Matching Contrastive Learning for Partially View-Aligned Clustering](https://mlsanity.com/paper/2512.15396/) (2025)

## Abstract (arXiv)

In recent several years, the information bottleneck (IB) principle provides an information-theoretic framework for deep multi-view clustering (MVC) by compressing multi-view observations while preserving the relevant information of multiple views. Although existing IB-based deep MVC methods have achieved huge success, they rely on variational approximation and distribution assumption to estimate the lower bound of mutual information, which is a notoriously hard and impractical problem in high-dimensional multi-view spaces. In this work, we propose a new differentiable information bottleneck (DIB) method, which provides a deterministic and analytical MVC solution by fitting the mutual information without the necessity of variational approximation. Specifically, we first propose to directly fit the mutual information of high-dimensional spaces by leveraging normalized kernel Gram matrix, which does not require any auxiliary neural estimator to estimate the lower bound of mutual information. Then, based on the new mutual information measurement, a deterministic multi-view neural network with analytical gradients is explicitly trained to parameterize IB principle, which derives a deterministic compression of input variables from different views. Finally, a triplet consistency discovery mechanism is devised, which is capable of mining the feature consistency, cluster consistency and joint consistency based on the deterministic and compact representations. Extensive experimental results show the superiority of our DIB method on 6 benchmarks compared with 13 state-of-the-art baselines.