RE-TRIANGLE: Does TRIANGLE Enable Multimodal Alignment Beyond Cosine Similarity in Retrieval?
Not on the map. Why?
Abstract (arXiv)
Multimodal alignment is critical for bridging the semantic gap in information retrieval. However, traditional pairwise strategies introduce a geometric blind spot: while they align anchor modalities (e.g., text) with others, they lack constraints to enforce mutual consistency between peripheral modalities (e.g., video and audio). The TRIANGLE framework addresses this by minimizing the area of modality triplets on a hypersphere to enforce holistic alignment. In this reproducibility study, we verify the robustness of this geometric objective for retrieval tasks. We confirm that TRIANGLE outperforms pairwise baselines in zero-shot settings, achieving Recall@1 gains of up to +8.7 points, though benefits are domain-dependent. However, we fail to reproduce the reported learning-from-scratch results. Analysis using a synthetic toy dataset attributes this to instability when jointly optimizing geometric alignment with Data-Text Matching (DTM) loss. Furthermore, we find that cosine regularization primarily stabilizes text-to-video retrieval, and fine-tuning with domain supervision amplifies geometric benefits but reduces cross-dataset generalization. Our findings support the efficacy of geometric alignment while highlighting critical optimization sensitivities. Code available at https://github.com/ARIJIT00171/RE-TRIANGLE.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 25 references that are in the map. First posted on arXiv 22 May 2026.
Builds on
15 of 25 references.
References · 15
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2017Localizing Moments in Video with Natural Languagecited by 663
- 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
- 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
- 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
- 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
- 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 169
- 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 110
- 2024Gramian Multimodal Representation Learning and Alignmentcited by 54
- 2024Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalitiescited by 24
3 more
- 2025A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similaritycited by 18
- 2024Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metriccited by 6
- 2025Reproducibility, Replicability, and Insights into Visual Document Retrieval with Late Interactioncited by 4
Built on by
Not cited yet.