# Balanced Multi-view Clustering

arXiv: https://arxiv.org/abs/2501.02564 · HTML: https://mlsanity.com/paper/2501.02564/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 94 references that are in the map.
- First posted on arXiv 5 Jan 2025.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Balanced Multimodal Learning via On-the-fly Gradient Modulation](https://mlsanity.com/paper/2203.15332/) (2022)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Honeybee: Locality-enhanced Projector for Multimodal LLM](https://mlsanity.com/paper/2312.06742/) (2023)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)](https://mlsanity.com/paper/2203.12221/) (2022)
- [PMR: Prototypical Modal Rebalance for Multimodal Learning](https://mlsanity.com/paper/2211.07089/) (2022)
- [Multi-level Feature Learning for Contrastive Multi-view Clustering](https://mlsanity.com/paper/2106.11193/) (2021)
- [Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models](https://mlsanity.com/paper/2410.03577/) (2024)
- [Large-scale Multi-view Subspace Clustering in Linear Time](https://mlsanity.com/paper/1911.09290/) (2019)
- [ReconBoost: Boosting Can Achieve Modality Reconcilement](https://mlsanity.com/paper/2405.09321/) (2024)
- [On Uni-Modal Feature Learning in Supervised Multi-Modal Learning](https://mlsanity.com/paper/2305.01233/) (2023)
- [Deep Multiview Clustering by Contrasting Cluster Assignments](https://mlsanity.com/paper/2304.10769/) (2023)
- [On-the-fly Modulation for Balanced Multimodal Learning](https://mlsanity.com/paper/2410.11582/) (2024)
- [Deep Embedded Multi-view Clustering with Collaborative Training](https://mlsanity.com/paper/2007.13067/) (2020)
- [A deep matrix factorization method for learning attribute representations](https://mlsanity.com/paper/1509.03248/) (2015)
- [Multi-View Spectral Clustering via Structured Low-Rank Matrix Factorization](https://mlsanity.com/paper/1709.01212/) (2017)
- [Investigating and Mitigating the Side Effects of Noisy Views for Self-Supervised Clustering Algorithms in Practical Multi-View Scenarios](https://mlsanity.com/paper/2303.17245/) (2023)
- [DealMVC: Dual Contrastive Calibration for Multi-view Clustering](https://mlsanity.com/paper/2308.09000/) (2023)
- [Improving Multimodal Learning with Multi-Loss Gradient Modulation](https://mlsanity.com/paper/2405.07930/) (2024)
- [Efficient Multi-view Clustering via Unified and Discrete Bipartite Graph Learning](https://mlsanity.com/paper/2209.04187/) (2022)

## Abstract (arXiv)

Multi-view clustering (MvC) aims to integrate information from different views to enhance the capability of the model in capturing the underlying data structures. The widely used joint training paradigm in MvC is potentially not fully leverage the multi-view information, since the imbalanced and under-optimized view-specific features caused by the uniform learning objective for all views. For instance, particular views with more discriminative information could dominate the learning process in the joint training paradigm, leading to other views being under-optimized. To alleviate this issue, we first analyze the imbalanced phenomenon in the joint-training paradigm of multi-view clustering from the perspective of gradient descent for each view-specific feature extractor. Then, we propose a novel balanced multi-view clustering (BMvC) method, which introduces a view-specific contrastive regularization (VCR) to modulate the optimization of each view. Concretely, VCR preserves the sample similarities captured from the joint features and view-specific ones into the clustering distributions corresponding to view-specific features to enhance the learning process of view-specific feature extractors. Additionally, a theoretical analysis is provided to illustrate that VCR adaptively modulates the magnitudes of gradients for updating the parameters of view-specific feature extractors to achieve a balanced multi-view learning procedure. In such a manner, BMvC achieves a better trade-off between the exploitation of view-specific patterns and the exploration of view-invariance patterns to fully learn the multi-view information for the clustering task. Finally, a set of experiments are conducted to verify the superiority of the proposed method compared with state-of-the-art approaches on eight benchmark MvC datasets.