mlsanityData to 14 Oct 2026

Scalable Vision Language Model Training via High Quality Data Curation

arXiv 2501.05952 ↗First posted 10 Jan 2025cs.CV cs.CLcited by 55

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

In this paper, we introduce SAIL-VL (ScAlable Vision Language Model TraIning via High QuaLity Data Curation), an open-source vision language model (VLM) series achieving state-of-the-art (SOTA) performance in 2B and 8B parameters. The following three key improvements contribute to SAIL-VL's leading performance: (1) Scalable high-quality visual understanding data construction: We implement a data construction pipeline to enable hundred-million-scale high-quality recaption data annotation. The resulted dataset SAIL-Caption is validated to be of the highest data quality compared with opensource datasets. (2) Scalable Pretraining with High-Quality Visual Understanding Data: We scale SAIL-VL's pretraining budget up to 655B tokens and show that even a 2B VLM benefits from scaled up training data sizes, exhibiting logarithmic data size scaling laws in benchmark performance. (3) Scalable SFT via data quantity and complexity scaling: We curate a high-quality SFT dataset collection with leading data quantity scaling effectiveness and demonstrate that training with progressively higher-complexity data surpasses baseline one-stage training by a large margin. SAIL-VL series models achieve the highest average score in 18 widely used VLM benchmarks in our evaluation, with the 2B model takes the top position over VLMs of comparable sizes on OpenCompass 2024 (https://rank.opencompass.org.cn/leaderboard-multimodal), demonstrating robust visual comprehension abilities. SAIL-VL series models are released at HuggingFace (https://huggingface.co/BytedanceDouyinContent).

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 78 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

44 of 78 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering30 Sep
  2. 2024CogVLM2: Visual Language Models for Image and Video Understanding29 Aug
  3. 2023CapsFusion: Rethinking Image-Text Data at Scale31 Oct
  4. 2023PaLI-3 Vision Language Models: Smaller, Faster, Stronger13 Oct
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  6. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts3 Nov

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023Segment Anythingcited by 8,234
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  12. 2019Towards VQA Models That Can Readcited by 1,718
26 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  7. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  8. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  10. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  11. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  12. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  13. 2016A Diagram Is Worth A Dozen Imagescited by 804
  14. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  15. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  16. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  17. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  18. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  19. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  20. 2019Scene Text Visual Question Answeringcited by 331
  21. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  22. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 87
  23. 2024Benchmarking and Improving Detail Image Captioncited by 81
  24. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Datacited by 67
  25. 2022OCR-IDL: OCR Annotations for Industry Document Library Datasetcited by 35
  26. 2020DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understandingcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 37; 2026: 18.

55 papers.

Sort:

By area:

  1. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
  2. 2025VGR: Visual Grounded Reasoningcited by 73
  3. 2025Ovis-U1 Technical Reportcited by 67
  4. 2025MindJourney: Test-Time Scaling with World Models for Spatial Reasoningcited by 50
  5. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  6. 2025WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild?cited by 26
  7. 2025Scaling Pre-training to One Hundred Billion Data for Vision Language Modelscited by 19
  8. 2025Compositional Generative Model of Unbounded 4D Citiescited by 17
  9. 2025M2-Reasoning: Empowering MLLMs with Unified General and Spatial Reasoningcited by 16
  10. 2025Co-Reinforcement Learning for Unified Multimodal Understanding and Generationcited by 15
  11. 2025MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMscited by 15
  12. 2025Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Modelscited by 14
38 more
  1. 2025SAIL-VL2 Technical Reportcited by 14
  2. 2025Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Modelscited by 11
  3. 2025Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resourcescited by 10
  4. 2025SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Modelcited by 10
  5. 2025Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMscited by 7
  6. 2025Affordance Benchmark for MLLMscited by 5
  7. 2025Describe Anything Model for Visual Question Answering on Text-rich Imagescited by 5
  8. 2025CrossHOI-Bench: A Unified Benchmark for HOI Evaluation across Vision-Language Models and HOI-Specific Methodscited by 5
  9. 2026ELVA: Exploring Ranking-Driven Universal Multimodal Retrievalcited by 5
  10. 2025Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMscited by 4
  11. 2025Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughputcited by 4
  12. 2025Guiding Cross-Modal Representations with MLLM Priors via Preference Alignmentcited by 4
  13. 2025SAILViT: Towards Robust and Generalizable Visual Backbones for MLLMs via Gradual Feature Refinementcited by 4
  14. 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
  15. 2025Diagnosing Corruption-Induced Reliability Failures in Vision-Language Modelscited by 4
  16. 2026Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillationcited by 4
  17. 2025GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devicescited by 3
  18. 2025AdaLRS: Loss-Guided Adaptive Learning Rate Search for Efficient Foundation Model Pretrainingcited by 3
  19. 2025AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomycited by 3
  20. 2025Towards Efficient Multimodal Unified Reasoning Model via Model Mergingcited by 3
  21. 2026Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translationcited by 3
  22. 2025Evaluating Small Vision-Language Models on Distance-Dependent Traffic Perceptioncited by 2
  23. 2025Discovering Intersectional Bias via Directional Alignment in Face Recognition Embeddingscited by 2
  24. 2025SAIL-RL: Guiding MLLMs in When and How to Think via Dual-Reward RL Tuningcited by 2
  25. 2026ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understandingcited by 2
  26. 2026S-GRPO: Unified Post-Training for Large Vision-Language Modelscited by 2
  27. 2026Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Modelscited by 2
  28. 2025GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMscited by 1
  29. 2025TowerVision: Understanding and Improving Multilinguality in Vision-Language Modelscited by 1
  30. 2025DenseAnnotate: Enabling Scalable Dense Caption Collection for Images and 3D Scenes via Spoken Descriptionscited by 1
  31. 2025SocialFusion: Addressing Social Degradation in Pre-trained Vision-Language Modelscited by 1
  32. 2026MaD-Mix: Multi-Modal Data Mixtures via Latent Space Coupling for Vision-Language Model Trainingcited by 1
  33. 2026Reinforced Curriculum Pre-Alignment for Domain-Adaptive VLMscited by 1
  34. 2026An Alternative Trajectory for Generative AIcited by 1
  35. 2026Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expandingcited by 1
  36. 2026Benchmark Shadows: Data Alignment, Parameter Footprints, and Generalization in Large Language Modelscited by 0
  37. 2026FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Modelscited by 0
  38. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Data Metabolism: An Efficient Data Design Schema For Vision Language Model
  2. 202620/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
  3. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Models
  4. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models
  5. 2023CiT: Curation in Training for Effective Vision-Language Data
  6. 2024DeepSeek-VL: Towards Real-World Vision-Language Understanding
  7. 2025The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer
  8. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models