# OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment

arXiv: https://arxiv.org/abs/2605.29900 · HTML: https://mlsanity.com/paper/2605.29900/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text](https://mlsanity.com/paper/2104.11178/) (2021)
- [Self-Supervised MultiModal Versatile Networks](https://mlsanity.com/paper/2006.16228/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset](https://mlsanity.com/paper/2305.18500/) (2023)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [MultiBench: Multiscale Benchmarks for Multimodal Representation Learning](https://mlsanity.com/paper/2107.07502/) (2021)
- [VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset](https://mlsanity.com/paper/2304.08345/) (2023)
- [Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos](https://mlsanity.com/paper/2104.12671/) (2021)
- [Gramian Multimodal Representation Learning and Alignment](https://mlsanity.com/paper/2412.11959/) (2024)
- [Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations](https://mlsanity.com/paper/2210.17444/) (2022)
- [InfoNCE: Identifying the Gap Between Theory and Practice](https://mlsanity.com/paper/2407.00143/) (2024)
- [OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces](https://mlsanity.com/paper/2407.11895/) (2024)
- [Multimodal Sensor Fusion with Differentiable Filters](https://mlsanity.com/paper/2010.13021/) (2020)
- [Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities](https://mlsanity.com/paper/2411.01053/) (2024)
- [A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity](https://mlsanity.com/paper/2509.24734/) (2025)
- [Learning Optimal Multimodal Information Bottleneck Representations](https://mlsanity.com/paper/2505.19996/) (2025)
- [Aligning Multimodal Representations through an Information Bottleneck](https://mlsanity.com/paper/2506.04870/) (2025)
- [Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning](https://mlsanity.com/paper/2506.04411/) (2025)
- [PCA++: How Uniformity Induces Robustness to Background Noise in Contrastive Learning](https://mlsanity.com/paper/2511.12278/) (2025)
- [Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning](https://mlsanity.com/paper/2504.21375/) (2025)

## Abstract (arXiv)

Contrastive learning is effective for aligning paired views or modalities, but alignment beyond two modalities remains non-trivial and comparatively underexplored. Pairwise CLIP-style losses decompose multi-modal alignment into independent two-way comparisons and therefore do not explicitly model higher-order dependencies among multiple modalities. Recent beyond-pairwise objectives approach this problem from statistical or geometric perspectives, but arbitrary-modality alignment still lacks a principled criterion for defining what each modality should preserve and compress relative to the others. We revisit arbitrary-modality alignment through the Information Bottleneck principle. In multi-modal learning, sufficiency should preserve information predictable from the remaining modalities, while minimality should compress modality-specific information not supported by them. This naturally leads to a One-vs-All view, where each modality is characterized with respect to the remaining modalities. We propose OVA-IB, an Information Bottleneck framework for arbitrary-modality alignment. OVA-IB optimizes a tractable One-vs-All contrastive lower bound for sufficiency connected to a Dual Total Correlation-style objective, uses a parameter-free geometry-aware projection score, and derives a tractable upper-bound regularizer for minimality by bounding each representation's dependence on its own input with representation distributions induced by the remaining modalities. Experiments on classification, regression, modality-agnostic evaluation, and cross-modal retrieval benchmarks demonstrate strong and robust performance.