# COBRA: Contrastive Bi-Modal Representation Algorithm

arXiv: https://arxiv.org/abs/2005.03687 · HTML: https://mlsanity.com/paper/2005.03687/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 12 of 12)

- [SuS-X: Training-Free Name-Only Transfer of Vision-Language Models](https://mlsanity.com/paper/2211.16198/) (2022)
- [No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance](https://mlsanity.com/paper/2404.04125/) (2024)
- [Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2109.01797/) (2021)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning](https://mlsanity.com/paper/2305.09299/) (2023)
- [Fake News Detection with Heterogeneous Transformer](https://mlsanity.com/paper/2205.03100/) (2022)
- [ArcSin: Adaptive ranged cosine Similarity injected noise for Language-Driven Visual Tasks](https://mlsanity.com/paper/2402.17298/) (2024)
- [Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2408.16029/) (2024)
- [Curriculum Learning Meets Weakly Supervised Modality Correlation Learning](https://mlsanity.com/paper/2212.07619/) (2022)
- [Labeling Comic Mischief Content in Online Videos with a Multimodal Hierarchical-Cross-Attention Model](https://mlsanity.com/paper/2406.07841/) (2024)
- [CMAL: A Novel Cross-Modal Associative Learning Framework for Vision-Language Pre-Training](https://mlsanity.com/paper/2410.12595/) (2024)
- [Multimodal Learning with Augmentation Techniques for Natural Disaster Assessment](https://mlsanity.com/paper/2511.00004/) (2025)

## Abstract (arXiv)

There are a wide range of applications that involve multi-modal data, such as cross-modal retrieval, visual question-answering, and image captioning. Such applications are primarily dependent on aligned distributions of the different constituent modalities. Existing approaches generate latent embeddings for each modality in a joint fashion by representing them in a common manifold. However these joint embedding spaces fail to sufficiently reduce the modality gap, which affects the performance in downstream tasks. We hypothesize that these embeddings retain the intra-class relationships but are unable to preserve the inter-class dynamics. In this paper, we present a novel framework COBRA that aims to train two modalities (image and text) in a joint fashion inspired by the Contrastive Predictive Coding (CPC) and Noise Contrastive Estimation (NCE) paradigms which preserve both inter and intra-class relationships. We empirically show that this framework reduces the modality gap significantly and generates a robust and task agnostic joint-embedding space. We outperform existing work on four diverse downstream tasks spanning across seven benchmark cross-modal datasets.