mlsanityData to 14 Oct 2026

Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis

arXiv 2503.08741 ↗First posted 11 Mar 2025cs.CV cs.AIcited by 14

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

The success of multi-modal large language models (MLLMs) has been largely attributed to the large-scale training data. However, the training data of many MLLMs is unavailable due to privacy concerns. The expensive and labor-intensive process of collecting multi-modal data further exacerbates the problem. Is it possible to synthesize multi-modal training data automatically without compromising diversity and quality? In this paper, we propose a new method, Oasis, to synthesize high-quality multi-modal data with only images. Oasis breaks through traditional methods by prompting only images to the MLLMs, thus extending the data diversity by a large margin. Our method features a delicate quality control method which ensures the data quality. We collected over 500k data and conducted incremental experiments on LLaVA-NeXT. Extensive experiments demonstrate that our method can significantly improve the performance of MLLMs. The image-based synthesis also allows us to focus on the specific-domain ability of MLLMs. Code and dataset are publicly available at https://github.com/Letian2003/MM_INF.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 58 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

37 of 58 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  3. 2024MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct9 Sep
  4. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  5. 2024DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception11 Jul
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
  8. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension25 Apr
  9. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark7 Feb
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  11. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  12. 2023Improved Baselines with Visual Instruction Tuning5 Oct
3 more
  1. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  3. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  8. 2024Qwen2 Technical Reportcited by 2,120
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
10 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  3. 2016A Diagram Is Worth A Dozen Imagescited by 804
  4. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  5. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  6. 2021InfographicVQAcited by 437
  7. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  8. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Modelscited by 205
  9. 2024MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversitycited by 38
  10. 2024VILA2: VILA Augmented VILAcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 9.

14 papers.

Sort:

By area:

  1. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  2. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 29
  3. 2025Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Startcited by 5
  4. 2026Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligencecited by 4
  5. 2026Synthetic Interaction Data for Scalable Personalization in Large Language Modelscited by 4
  6. 2025ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructionscited by 3
  7. 2026DataEvolver: Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generationcited by 3
  8. 2025PlanGPT-VL: Enhancing Urban Planning with Domain-Specific Vision-Language Modelscited by 2
  9. 2025Aether Weaver: Multimodal Affective Narrative Co-Generation with Dynamic Scene Graphscited by 1
  10. 2026Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuningcited by 1
  11. 2026Failure-Informed Image Self-Augmentation for Multimodal Large Language Model Self-Improvementcited by 1
  12. 2026SecProbe: Adaptive Evaluation of Coding Agents on Cybersecurity Vulnerabilitiescited by 1
2 more
  1. 2026R1-SyntheticVL: Is Synthetic Data from Generative Models Ready for Multimodal Large Language Model?cited by 0
  2. 2026ReMem: Streaming Video Understanding With Long Context Retentioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data
  2. 2025Text-Only Data Synthesis for Vision Language Model Training
  3. 2024On Domain-Adaptive Post-Training for Multimodal Large Language Models
  4. 2025Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuning
  5. 2023Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels?
  6. 2023MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets
  7. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs
  8. 2023Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator