Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Developing vision-language models (VLMs) that generalize across diverse tasks requires large-scale training datasets with diverse content. In English, such datasets are typically constructed by aggregating and curating numerous existing visual question answering (VQA) resources. However, this strategy does not readily extend to other languages, where VQA datasets remain limited in both scale and domain coverage. In this work, we introduce Jagle, the largest Japanese multimodal post-training dataset to date, comprising approximately 9.2 million instances across diverse tasks. Rather than relying on existing VQA datasets, we collect heterogeneous source data, including images, image-text pairs, and PDF documents, and generate VQA pairs through multiple strategies such as VLM-based QA generation, translation, and text rendering. Experiments demonstrate that a 2.2B model trained with Jagle surpasses InternVL3.5-2B in average score across ten Japanese evaluation tasks and comes within five points of Qwen3-VL-2B-Instruct. Moreover, combining Jagle with FineVision improves English performance over training with FineVision alone. These benefits generalize across model backbones and scales: LLM-jp-4-VL 9B beta, a 9B-scale model built on the Japanese-centric LLM-jp-4-8B-instruct, achieves Japanese performance comparable to Qwen3-VL-8B-Instruct. To facilitate reproducibility and future research, we release the dataset, models, and code.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 63 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
45 of 63 references.
Closest ancestors · 7
Same direction, newest first
- 2026JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation1 Apr
- 2025FineVision: Open Data Is All You Need20 Oct
- 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models20 Jan
- 2024JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation22 Oct
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models25 Sep
- 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark10 Jun
- 2024Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese11 Apr
Other references · 38
- 2023Visual Instruction Tuningcited by 7,792
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024GPT-4o System Cardcited by 4,467
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2019Towards VQA Models That Can Readcited by 1,696
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
26 more
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2023VILA: On Pre-training for Visual Language Modelscited by 686
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
- 2021InfographicVQAcited by 430
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
- 2019PlotQA: Reasoning over Scientific Plotscited by 317
- 2024Evolutionary Optimization of Model Merging Recipescited by 204
- 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 92
- 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 84
- 2024CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacycited by 80
- 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 54
- 2025NVIDIA Nemotron 3: Efficient and Open Intelligencecited by 54
- 2024LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMscited by 31
- 2025NVIDIA Nemotron Nano V2 VLcited by 15
- 2026Olmix: A Framework for Data Mixing Throughout LM Developmentcited by 12
- 2024JDocQA: Japanese Document Question Answering Dataset for Generative Language Modelscited by 9
- 2024Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Modelcited by 6
- 2025Evaluating Multimodal Large Language Models on Vertically Written Japanese Textcited by 4
- 2025DEJIMA: A Novel Large-scale Japanese Dataset for Image Captioning and Visual Question Answeringcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types
- 2025KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Language
- 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
- 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models
- 2022cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation
- 2024VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation
- 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
- 2024Visual Question Answering Instruction: Unlocking Multimodal Large Language Model To Domain-Specific Visual Multitasks