mlsanityData to 7 Oct 2026

LongProLIP: A Probabilistic Vision-Language Model with Long Context Text

arXiv 2503.08048 ↗First posted 11 Mar 2025cs.CV cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Recently, Probabilistic Language-Image Pre-Training (ProLIP) has been proposed to tackle the multiplicity issue of vision-language (VL) tasks. Despite their success in probabilistic representation learning at a scale, the ProLIP models cannot handle long context texts longer than 64 context length, which limits their ability to capture rich contextual information from longer text sequences. To address this issue, this paper proposes a fine-tuning strategy for ProLIP to accept longer texts, e.g., 256 text tokens. Experimental results on Urban-1k and the DataComp evaluation suite show that the proposed LongProLIP recipe can improve understanding of long contexts while minimizing the negative effect of fine-tuning.We also observe a trade-off between the long context understanding (measured by Urban-1k) and general zero-shot capability (measured by evaluation datasets by DataComp). Code is available at https://github.com/naver-ai/prolip

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 46 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

28 of 46 references.

References · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2013Describing Textures in the Wildcited by 2,110
  5. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  6. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  7. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  8. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  10. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
16 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  2. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  3. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  4. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  5. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  6. 2017Functional Map of the Worldcited by 338
  7. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 298
  8. 2023Data Filtering Networkscited by 247
  9. 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 150
  10. 2021The iWildCam 2021 Competition Datasetcited by 77
  11. 2023GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognitioncited by 50
  12. 2022MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Modelcited by 41
  13. 2023Improved Probabilistic Image-Text Representationscited by 40
  14. 2022ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCOcited by 37
  15. 2024HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Textscited by 17
  16. 2024Probabilistic Language-Image Pre-Trainingcited by 16

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learningcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
  2. 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly
  3. 2024Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models
  4. 2024DreamLIP: Language-Image Pre-training with Long Captions
  5. 2026How to Train Your Long-Context Visual Document Model
  6. 2023ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models
  7. 2025LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models
  8. 2025MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models