mlsanityData to 7 Oct 2026

ViCLIP-OT: The First Foundation Vision-Language Model for Vietnamese Image-Text Retrieval with Optimal Transport

arXiv 2602.22678 ↗First posted 26 Feb 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Image-text retrieval has become a fundamental component in intelligent multimedia systems; however, most existing vision-language models are optimized for highresource languages and remain suboptimal for low-resource settings such as Vietnamese. This work introduces ViCLIP-OT, a foundation vision-language model specifically designed for Vietnamese image-text retrieval. The proposed framework integrates CLIP-style contrastive learning with a Similarity-Graph Regularized Optimal Transport (SIGROT) loss to enhance global cross-modal consistency and mitigate modality gap issues. Extensive experiments on three Vietnamese benchmarks (UITOpenViIC, KTVIC, and Crossmodal-3600) demonstrate that ViCLIP-OT consistently outperforms CLIP and SigLIP baselines in both in-domain and zero-shot settings. On UIT-OpenViIC, the model achieves an average Recall@K of 67.34%, improving upon CLIP by 5.75 percentage points. In zero-shot evaluation on Crossmodal-3600, ViCLIPOT surpasses CLIP by 11.72 percentage points. Embedding-space analysis further confirms improved alignment and reduced modality gap. The results indicate that integrating SIGROT provides an effective and scalable strategy for cross-modal retrieval in low-resource languages, offering practical implications for intelligent multimedia retrieval systems in Vietnamese and other underrepresented linguistic contexts.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 44 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

29 of 44 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  8. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  10. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  11. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  12. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
17 more
  1. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  2. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  3. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  4. 2015Learning with a Wasserstein Losscited by 194
  5. 2022A Self-Supervised Descriptor for Image Copy Detectioncited by 175
  6. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  7. 2022CyCLIP: Cyclic Contrastive Language-Image Pretrainingcited by 127
  8. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  9. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  10. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  11. 2023Recent Advances in Optimal Transport for Machine Learningcited by 66
  12. 2023Scalable Optimal Transport Methods in Machine Learning: A Contemporary Surveycited by 27
  13. 2024Explaining and Mitigating the Modality Gap in Contrastive Multimodal Learningcited by 25
  14. 2020UIT-ViIC: A Dataset for the First Evaluation on Vietnamese Image Captioningcited by 13
  15. 2025ELIP: Enhanced Visual-Language Foundation Models for Image Retrievalcited by 9
  16. 2024Optimizing CLIP Models for Image Retrieval with Maintained Joint-Embedding Alignmentcited by 6
  17. 2025Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task
  2. 2025uCLIP: Parameter-Efficient Multilingual Extension of Vision-Language Models with Unpaired Data
  3. 2024RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training
  4. 2025Vision-Free Retrieval: Rethinking Multimodal Search with Textual Scene Descriptions
  5. 2024Toward Automatic Relevance Judgment using Vision--Language Models for Image--Text Retrieval Evaluation
  6. 2022HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval
  7. 2025Adding simple structure at inference improves Vision-Language Compositionality
  8. 2025VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark