mlsanityData to 7 Oct 2026

EVA-CLIP: Improved Training Techniques for CLIP at Scale

arXiv 2303.15389 ↗First posted 27 Mar 2023cs.CVcited by 786

Not on the map. Why?

Cited by 329 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Contrastive language-image pre-training, CLIP for short, has gained increasing attention for its potential in various scenarios. In this paper, we propose EVA-CLIP, a series of models that significantly improve the efficiency and effectiveness of CLIP training. Our approach incorporates new techniques for representation learning, optimization, and augmentation, enabling EVA-CLIP to achieve superior performance compared to previous CLIP models with the same number of parameters but significantly smaller training costs. Notably, our largest 5.0B-parameter EVA-02-CLIP-E/14+ with only 9 billion seen samples achieves 82.0 zero-shot top-1 accuracy on ImageNet-1K val. A smaller EVA-02-CLIP-L/14+ with only 430 million parameters and 6 billion seen samples achieves 80.4 zero-shot top-1 accuracy on ImageNet-1K val. To facilitate open access and open research, we release the complete suite of EVA-CLIP to the community at https://github.com/baaivision/EVA/tree/master/EVA-CLIP.

Key facts. Cited by 786 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

30 of 51 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  12. 2013Describing Textures in the Wildcited by 2,110
18 more
  1. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  2. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  4. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  5. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  6. 2019Natural Adversarial Examplescited by 1,401
  7. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  8. 2016Deep Networks with Stochastic Depthcited by 973
  9. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  10. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  11. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  12. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 671
  13. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  14. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  15. 2018A Short Note about Kinetics-600cited by 331
  16. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  17. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  18. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 88; 2024: 265; 2025: 257; 2026: 176.

786 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  7. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  8. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  9. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  10. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  11. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  12. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
38 more
  1. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  2. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  3. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  4. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  5. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  7. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  8. 2024What matters when building vision-language models?cited by 321
  9. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  10. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  11. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  12. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  13. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  14. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  15. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  16. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  17. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
  18. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  19. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  20. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 192
  21. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
  22. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  23. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  24. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  25. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  26. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 167
  27. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  28. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  29. 2023Emu: Generative Pretraining in Multimodalitycited by 156
  30. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 154
  31. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  32. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
  33. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 143
  34. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 137
  35. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  36. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  37. 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
  38. 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questionscited by 125

50 most cited of 786 All 786 (JSON)