mlsanityData to 14 Oct 2026

WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models

arXiv 2308.10755 ↗First posted 21 Aug 2023cs.CL cs.CVcited by 50

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

The rise in popularity of ChatGPT and GPT-4 has significantly accelerated the development of large models, leading to the creation of numerous impressive large language models(LLMs) and multimodal large language models (MLLMs). These cutting-edge models owe their remarkable performance to high-quality data. However, the details of the training data used in leading paradigms are often kept confidential. This lack of transparency, coupled with the scarcity of open-source data, impedes further developments within the community. As a response, this paper presents "Wan Juan", a large-scale multimodal dataset composed of both Chinese and English data, collected from a wide range of web sources. The dataset incorporates text, image-text, and video modalities, with a total volume exceeding 2TB. It was utilized in the training of InternLM, a model that demonstrated significant advantages in multi-dimensional evaluations when compared to models of a similar scale. All data can be accessed at https://opendatalab.org.cn/WanJuan1.0.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 18 references that are in the map. First posted on arXiv 21 Aug 2023.

Builds on

12 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 11

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  10. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  11. 2023Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Textcited by 208

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 22; 2025: 11; 2026: 10.

50 papers.

Sort:

By area:

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  2. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  3. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  4. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  5. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  6. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  7. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
  8. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  9. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  10. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  11. 2024Memory3: Language Modeling with Explicit Memorycited by 40
  12. 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Modelscited by 34
38 more
  1. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  2. 2024OpenDataLab: Empowering General Artificial Intelligence with Open Datasetscited by 29
  3. 2024EVLM: An Efficient Vision-Language Model for Visual Understandingcited by 25
  4. 2024The Evolution of Multimodal Model Architecturescited by 21
  5. 2025Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Modelscited by 19
  6. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scalecited by 18
  7. 2023Ziya2: Data-centric Learning is All LLMs Needcited by 15
  8. 2024DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Modelscited by 15
  9. 2026TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Renderingcited by 15
  10. 2025Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Modelscited by 13
  11. 2023MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applicationscited by 11
  12. 2024LongWanjuan: Towards Systematic Measurement for Long Text Qualitycited by 11
  13. 2025SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversitycited by 10
  14. 2023YUAN 2.0: A Large Language Model with Localized Filtering-based Attentioncited by 9
  15. 2024SentenceVAE: Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Contextcited by 7
  16. 2024YuLan-Mini: An Open Data-efficient Language Modelcited by 7
  17. 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMscited by 6
  18. 2025MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languagescited by 5
  19. 2026LegalOne: A Family of Foundation Models for Reliable Legal Reasoningcited by 5
  20. 2023MC2: Towards Transparent and Culturally-Aware NLP for Minority Languages in Chinacited by 4
  21. 2024YuLan: An Open-source Large Language Modelcited by 4
  22. 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
  23. 2026Data Science and Technology Towards AGI Part I: Tiered Data Managementcited by 4
  24. 2023MiChao-HuaFen 1.0: A Specialized Pre-trained Corpus Dataset for Domain-specific Large Modelscited by 2
  25. 2024Enhancing Startup Success Predictions in Venture Capital: A GraphRAG Augmented Multivariate Time Series Methodcited by 2
  26. 2025PCMind-2.1-Kaiyuan-2B Technical Reportcited by 2
  27. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Modelscited by 2
  28. 2025CAPC-CG: A Large-Scale, Expert-Directed LLM-Annotated Corpus of Adaptive Policy Communication in Chinacited by 1
  29. 2026PodBench: A Comprehensive Benchmark for Instruction-Aware Audio-Oriented Podcast Script Generationcited by 1
  30. 2026Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within 5090cited by 1
  31. 2024ChuXin: 1.6B Technical Reportcited by 0
  32. 2024OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruningcited by 0
  33. 2025FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translationcited by 0
  34. 2025S2LM: Towards Semantic Steganography via Large Language Modelscited by 0
  35. 2026Auditing Chinese Web-scale Corpora via Sampled BPE Token Statisticscited by 0
  36. 2026LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretrainingcited by 0
  37. 2026LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-trainingcited by 0
  38. 2026Dynamic Semantic Compression for Efficient Latent-Space Inference in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective
  2. 2023Multimodal Large Language Models: A Survey
  3. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
  5. 2023Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages
  6. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Models
  7. 2024X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment
  8. 2024MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish