mlsanityData to 14 Oct 2026

On the Geometry of Multimodal Saturation: Riemannian VICReg

arXiv 2610.06096 ↗First posted 5 Oct 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In self-supervised learning, a third modality should improve, or at least preserve, performance. Across nine image-text-tabular datasets, we show that it instead harms performance: the trimodal model underperforms its own best bimodal subset in 55.6% of paired runs under VICReg. The same failure occurs in 51.1% of paired runs under SimSiam. We call this failure multimodal saturation. We propose that the failure lies in the alignment geometry. Riemannian VICReg (R-VICReg) generalizes classical VICReg: it aligns views by squared geodesic distance on learnable negative-curvature product factors and recovers VICReg exactly as curvature vanishes. Over the same 45 paired runs, R-VICReg raises the probability that the third modality helps from 44.4% to 64.4%, with gains concentrated where VICReg saturates.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

31 of 45 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  5. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  6. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  7. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  10. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  11. 2018Hyperbolic Neural Networkscited by 402
  12. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
19 more
  1. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
  2. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
  3. 2023Hyperbolic Image-Text Representationscited by 136
  4. 2022Multimodal Masked Autoencoders Learn Transferable Representationscited by 109
  5. 2019Constant Curvature Graph Convolutional Networkscited by 108
  6. 2022Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networkscited by 95
  7. 2019Mixed-curvature Variational Autoencoderscited by 86
  8. 2022Hyperbolic Contrastive Learning for Visual Representations beyond Objectscited by 61
  9. 2023HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representationscited by 44
  10. 2025A Closer Look at Multimodal Representation Collapsecited by 28
  11. 2023LANISTR: Multimodal Learning from Structured and Unstructured Datacited by 11
  12. 2013Efficient Classification for Metric Datacited by 10
  13. 2022Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representationscited by 10
  14. 2025BalanceBenchmark: A Survey for Multimodal Imbalance Learningcited by 8
  15. 2023MuG: A Multimodal Classification Benchmark on Game Data with Tabular, Textual, and Visual Fieldscited by 6
  16. 2023HMSN: Hyperbolic Self-Supervised Learning by Clustering with Ideal Prototypescited by 6
  17. 2025Negative to Positive Co-learning with Aggressive Modality Dropoutcited by 3
  18. 2025Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Spacecited by 3
  19. 2025Self-Supervised Learning by Curvature Alignmentcited by 1

Built on by

Not cited yet.