mlsanityData to 7 Oct 2026

Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languages

arXiv 2603.23521 ↗First posted 6 Mar 2026cs.CL cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal research has predominantly focused on single-image reasoning, with limited exploration of multi-image scenarios. Recent models have sought to enhance multi-image understanding through large-scale pretraining on interleaved image-text datasets. However, most Vision-Language Models (VLMs) are trained primarily on English datasets, leading to inadequate representation of Indian languages. To address this gap, we introduce the Chitrakshara dataset series, covering 11 Indian languages sourced from Common Crawl. It comprises (1) Chitrakshara-IL, a large-scale interleaved pretraining dataset with 193M images, 30B text tokens, and 50M multilingual documents, and (2) Chitrakshara-Cap, which includes 44M image-text pairs with 733M tokens. This paper details the data collection pipeline, including curation, filtering, and processing methodologies. Additionally, we present a comprehensive quality and diversity analysis to assess the dataset's representativeness across Indic languages and its potential for developing more culturally inclusive VLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 50 of its 67 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

50 of 67 references.

References · 50

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
38 more
  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  2. 2024Mixtral of Expertscited by 1,785
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  4. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  6. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  7. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  8. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  10. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  11. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  12. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  13. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  14. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  15. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  16. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  17. 2024InternLM2 Technical Reportcited by 391
  18. 2024What matters when building vision-language models?cited by 321
  19. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  20. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 230
  21. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  22. 2023Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Textcited by 208
  23. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  24. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 138
  25. 2024xGen-MM (BLIP-3): A Family of Open Large Multimodal Modelscited by 116
  26. 2024Benchmarking Vision Language Models for Cultural Understandingcited by 100
  27. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 63
  28. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 58
  29. 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
  30. 2024MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokenscited by 47
  31. 2020AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languagescited by 43
  32. 2024PALO: A Polyglot Large Multimodal Model for 5B Peoplecited by 34
  33. 2024Airavata: Introducing Hindi Instruction-tuned LLMcited by 32
  34. 2023Building a Llama2-finetuned LLM for Odia Language Utilizing Domain Knowledge Instruction Setcited by 12
  35. 2024SUTRA: Scalable Multilingual Language Model Architecturecited by 11
  36. 2025Chitrarth: Bridging Vision and Language for a Billion Peoplecited by 10
  37. 2024mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpuscited by 8
  38. 2025Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
  2. 2024Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models
  3. 2025Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation
  4. 2026Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems
  5. 2026Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
  6. 2024Maya: An Instruction Finetuned Multilingual Multimodal Model
  7. 2025Behind Maya: Building a Multilingual Vision Language Model
  8. 2025HinTel-AlignBench: A Framework and Benchmark for Hindi-Telugu with English-Aligned Samples