mlsanityData to 7 Oct 2026

EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters

arXiv 2402.04252 ↗First posted 6 Feb 2024cs.CVcited by 99

Not on the map. Why?

Cited by 39 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Scaling up contrastive language-image pretraining (CLIP) is critical for empowering both vision and multimodal models. We present EVA-CLIP-18B, the largest and most powerful open-source CLIP model to date, with 18-billion parameters. With only 6-billion training samples seen, EVA-CLIP-18B achieves an exceptional 80.7% zero-shot top-1 accuracy averaged across 27 widely recognized image classification benchmarks, outperforming its forerunner EVA-CLIP (5-billion parameters) and other open-source CLIP models by a large margin. Remarkably, we observe a consistent performance improvement with the model size scaling of EVA-CLIP, despite maintaining a constant training dataset of 2-billion image-text pairs from LAION-2B and COYO-700M. This dataset is openly available and much smaller than the in-house datasets (e.g., DFN-5B, WebLI-10B) employed in other state-of-the-art CLIP models. EVA-CLIP-18B demonstrates the potential of EVA-style weak-to-strong visual model scaling. With our model weights made publicly available, we hope to facilitate future research in vision and multimodal foundation models.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Builds on 52 of its 76 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

52 of 76 references.

References · 52

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2016Layer Normalizationcited by 5,579
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  7. 2013Describing Textures in the Wildcited by 2,110
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  9. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  10. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  13. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  14. 2019Root Mean Square Layer Normalizationcited by 1,425
  15. 2019Natural Adversarial Examplescited by 1,401
  16. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  17. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  18. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  19. 2016Deep Networks with Stochastic Depthcited by 973
  20. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  21. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  22. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  23. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  24. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  25. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 671
  26. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  27. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  28. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  29. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  30. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  31. 2018A Short Note about Kinetics-600cited by 331
  32. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  33. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  34. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  35. 2023Data Filtering Networkscited by 247
  36. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  37. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  38. 2023Emu: Generative Pretraining in Multimodalitycited by 156

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 37; 2026: 37.

99 papers.

Sort:

By area:

  1. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  2. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  3. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  4. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
  5. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  6. 2024Unveiling Encoder-Free Vision-Language Modelscited by 90
  7. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  8. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  9. 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
  10. 2025Scaling Language-Free Visual Representation Learningcited by 62
  11. 2024EchoSight: Advancing Visual-Language Models with Wiki Knowledgecited by 61
  12. 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
38 more
  1. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 50
  2. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoderscited by 49
  3. 2024Jina CLIP: Your CLIP Model Is Also Your Text Retrievercited by 46
  4. 2024Baichuan-Omni Technical Reportcited by 42
  5. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 41
  6. 2024OmniBind: Large-scale Omni Multimodal Representation via Binding Spacescited by 38
  7. 2025EVEv2: Improved Baselines for Encoder-Free Vision-Language Modelscited by 38
  8. 2024Diffusion Feedback Helps CLIP See Bettercited by 37
  9. 2024Beyond Bare Queries: Open-Vocabulary Object Grounding with 3D Scene Graphcited by 32
  10. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  11. 2024Label Propagation for Zero-shot Classification with Vision-Language Modelscited by 28
  12. 2025Breaking the Batch Barrier (B3) of Contrastive Learning via Smart Batch Miningcited by 27
  13. 2025Temporal Chain of Thought: Long-Video Understanding by Thinking in Framescited by 27
  14. 2025Improve Multi-Modal Embedding Learning via Explicit Hard Negative Gradient Amplifyingcited by 21
  15. 2024Augmenting Multimodal LLMs with Self-Reflective Tokens for Knowledge-based Visual Question Answeringcited by 20
  16. 2025MMKB-RAG: A Multi-Modal Knowledge-Based Retrieval-Augmented Generation Frameworkcited by 18
  17. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
  18. 2024VladVA: Discriminative Fine-tuning of LVLMscited by 15
  19. 2024HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluatorcited by 12
  20. 2025DIFFER: Disentangling Identity Features via Semantic Cues for Clothes-Changing Person Re-IDcited by 11
  21. 2025OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrievalcited by 10
  22. 2024Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Modelscited by 9
  23. 2024Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammographycited by 9
  24. 2024Scaling White-Box Transformers for Visioncited by 8
  25. 2024On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoningcited by 7
  26. 2024FastCLIP: A Suite of Optimization Techniques to Accelerate CLIP Training with Limited Resourcescited by 7
  27. 2025ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answeringcited by 7
  28. 2026TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrievalcited by 7
  29. 2026ELVA: Exploring Ranking-Driven Universal Multimodal Retrievalcited by 5
  30. 2025MASR: Self-Reflective Reasoning through Multimodal Hierarchical Attention Focusing for Agent-based Video Understandingcited by 4
  31. 2025mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generationcited by 4
  32. 2025SuperCLIP: CLIP with Simple Classification Supervisioncited by 4
  33. 2025LR0.FM: Low-Res Benchmark and Improving Robustness for Zero-Shot Classification in Foundation Modelscited by 3
  34. 2025Mind the (Data) Gap: Evaluating Vision Systems in Small Data Applicationscited by 3
  35. 2025Guiding Cross-Modal Representations with MLLM Priors via Preference Alignmentcited by 3
  36. 2025Diversity-Guided MLP Reduction for Efficient Large Vision Transformerscited by 3
  37. 2025MobileViCLIP: An Efficient Video-Text Model for Mobile Devicescited by 3
  38. 2025Enhancing 3D Semantic Scene Completion with a Refinement Modulecited by 3

50 most cited of 99 All 99 (JSON)