# Meta CLIP 2: A Worldwide Scaling Recipe

arXiv: https://arxiv.org/abs/2507.22062 · HTML: https://mlsanity.com/paper/2507.22062/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 52 of its 67 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 52)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [LiT: Zero-Shot Transfer with Locked-image text Tuning](https://mlsanity.com/paper/2111.07991/) (2021)
- [SLIP: Self-supervision meets Language-Image Pre-training](https://mlsanity.com/paper/2112.12750/) (2021)
- [DataComp-LM: In search of the next generation of training sets for language models](https://mlsanity.com/paper/2406.11794/) (2024)
- [Perception Encoder: The best visual embeddings are not at the output of the network](https://mlsanity.com/paper/2504.13181/) (2025)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Demystifying CLIP Data](https://mlsanity.com/paper/2309.16671/) (2023)

## Built on by (most cited first, 30 of 64)

- [Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception](https://mlsanity.com/paper/2602.11858/) (2026)
- [MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction](https://mlsanity.com/paper/2509.18095/) (2025)
- [FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model](https://mlsanity.com/paper/2510.10921/) (2025)
- [In Pursuit of Pixel Supervision for Visual Pre-training](https://mlsanity.com/paper/2512.15715/) (2025)
- [FireRed-OCR Technical Report](https://mlsanity.com/paper/2603.01840/) (2026)
- [TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment](https://mlsanity.com/paper/2604.12012/) (2026)
- [OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence](https://mlsanity.com/paper/2602.08683/) (2026)
- [The Information Geometry of Softmax: Probing and Steering](https://mlsanity.com/paper/2602.15293/) (2026)
- [Brought a Gun to a Knife Fight: Modern VFM Baselines Outgun Specialized Detectors on In-the-Wild AI Image Detection](https://mlsanity.com/paper/2509.12995/) (2025)
- [Revisiting Multilingual Data Mixtures in Language Model Pretraining](https://mlsanity.com/paper/2510.25947/) (2025)
- [SuperFlow: Training Flow Matching Models with RL on the Fly](https://mlsanity.com/paper/2512.17951/) (2025)
- [Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model](https://mlsanity.com/paper/2607.24904/) (2026)
- [WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models](https://mlsanity.com/paper/2602.02537/) (2026)
- [OpenVision 3: A Family of Unified Visual Encoder for Both Understanding and Generation](https://mlsanity.com/paper/2601.15369/) (2026)
- [Sapiens2](https://mlsanity.com/paper/2604.21681/) (2026)
- [ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge](https://mlsanity.com/paper/2602.09839/) (2026)
- [Beyond Language: Grounding Referring Expressions with Hand Pointing in Egocentric Vision](https://mlsanity.com/paper/2603.26646/) (2026)
- [Let ViT Speak: Generative Language-Image Pre-training](https://mlsanity.com/paper/2605.00809/) (2026)
- [On the dynamic evolution of CLIP texture-shape bias and its relationship to human alignment and model robustness](https://mlsanity.com/paper/2508.09814/) (2025)
- [GRAPE: Let GRPO Supervise Query Rewriting by Ranking for Retrieval](https://mlsanity.com/paper/2509.23370/) (2025)
- [Language as a Label: Zero-Shot Multimodal Classification of Everyday Postures under Data Scarcity](https://mlsanity.com/paper/2510.13364/) (2025)
- [Text-Guided Semantic Image Encoder](https://mlsanity.com/paper/2511.20770/) (2025)
- [PowerCLIP: Powerset Alignment for Contrastive Pre-Training](https://mlsanity.com/paper/2511.23170/) (2025)
- [EchoVLM: Measurement-Grounded Multimodal Learning for Echocardiography](https://mlsanity.com/paper/2512.12107/) (2025)
- [HEDGE: Heterogeneous Ensemble for Detection of AI-GEnerated Images in the Wild](https://mlsanity.com/paper/2604.03555/) (2026)
- [LOGER: Local--Global Ensemble for Robust Deepfake Detection in the Wild](https://mlsanity.com/paper/2604.03558/) (2026)
- [CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis](https://mlsanity.com/paper/2605.12650/) (2026)
- [Decoupling the Declarative from the Procedural in Vision-Language-Action Models](https://mlsanity.com/paper/2606.21496/) (2026)
- [On the Limitations of Cross-Lingual Consistency in Multilingual Text-to-image Generation](https://mlsanity.com/paper/2608.11002/) (2026)
- [Eigenfunction Extraction for Ordered Representation Learning](https://mlsanity.com/paper/2510.24672/) (2025)

## Abstract (arXiv)

Contrastive Language-Image Pretraining (CLIP) is a popular foundation model, supporting from zero-shot classification, retrieval to encoders for multimodal large language models (MLLMs). Although CLIP is successfully trained on billion-scale image-text pairs from the English world, scaling CLIP's training further to learning from the worldwide web data is still challenging: (1) no curation method is available to handle data points from non-English world; (2) the English performance from existing multilingual CLIP is worse than its English-only counterpart, i.e., "curse of multilinguality" that is common in LLMs. Here, we present Meta CLIP 2, the first recipe training CLIP from scratch on worldwide web-scale image-text pairs. To generalize our findings, we conduct rigorous ablations with minimal changes that are necessary to address the above challenges and present a recipe enabling mutual benefits from English and non-English world data. In zero-shot ImageNet classification, Meta CLIP 2 ViT-H/14 surpasses its English-only counterpart by 0.8% and mSigLIP by 0.7%, and surprisingly sets new state-of-the-art without system-level confounding factors (e.g., translation, bespoke architecture changes) on multilingual benchmarks, such as CVQA with 57.4%, Babel-ImageNet with 50.2% and XM3600 with 64.3% on image-to-text retrieval.