mlsanityData to 14 Oct 2026

LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training

arXiv 2509.23661 ↗First posted 28 Sep 2025cs.CVcited by 244

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

We present LLaVA-OneVision-1.5, a novel family of Large Multimodal Models (LMMs) that achieve state-of-the-art performance with significantly reduced computational and financial costs. Different from the existing works, LLaVA-OneVision-1.5 provides an open, efficient, and reproducible framework for building high-quality vision-language models entirely from scratch. The LLaVA-OneVision-1.5 release comprises three primary components: (1) Large-Scale Curated Datasets: We construct an 85M concept-balanced pretraining dataset LLaVA-OneVision-1.5-Mid-Traning and a meticulously curated 22M instruction dataset LLaVA-OneVision-1.5-Instruct. (2) Efficient Training Framework: We develop a complete end-to-end efficient training framework leveraging an offline parallel data packing strategy to facilitate the training of LLaVA-OneVision-1.5 within a 16,000 budget. (3) State-of-the-art Performance: Experimental results demonstrate that LLaVA-OneVision-1.5 yields exceptionally competitive performance across a broad range of downstream tasks. Specifically, LLaVA-OneVision-1.5-8B outperforms Qwen2.5-VL-7B on 18 of 27 benchmarks, and LLaVA-OneVision-1.5-4B surpasses Qwen2.5-VL-3B on all 27 benchmarks. (4) RL-based Post-training: We unlock the model's latent potential through a lightweight RL stage, effectively eliciting robust chain-of-thought reasoning to significantly boost performance on complex multimodal reasoning tasks.

Key facts. Cited by 244 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 59 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

55 of 59 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources1 Apr
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  4. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?23 Aug
  5. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  6. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models17 Jul
  7. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  8. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension25 Apr
  9. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  10. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension30 Jul
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun
  12. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents21 Jun

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2023Segment Anythingcited by 8,234
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
31 more
  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  4. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  5. 2016A Diagram Is Worth A Dozen Imagescited by 804
  6. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  8. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  9. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  10. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  11. 2021InfographicVQAcited by 437
  12. 2025Seed1.5-VL Technical Reportcited by 373
  13. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  14. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323
  15. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  16. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 286
  17. 2023Demystifying CLIP Datacited by 270
  18. 2023Data Filtering Networkscited by 250
  19. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  20. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  21. 2023Teaching CLIP to Count to Tencited by 198
  22. 2024Multimodal Autoregressive Pre-training of Large Vision Encoderscited by 109
  23. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 86
  24. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
  25. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
  26. 2022Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligencecited by 70
  27. 2024Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Modelscited by 53
  28. 2022CCMB: A Large-scale Chinese Cross-modal Benchmarkcited by 21
  29. 2025UniSVG: A Unified Dataset for Vector Graphic Understanding and Generation with Multimodal Large Language Modelscited by 15
  30. 2024Multi-label Cluster Discrimination for Visual Representation Learningcited by 14
  31. 2025Region-based Cluster Discrimination for Visual Representation Learningcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 38; 2026: 206.

244 papers.

Sort:

By area:

  1. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 65
  2. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 42
  3. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 40
  4. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  5. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 37
  6. 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipecited by 37
  7. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  8. 2026STEP3-VL-10B Technical Reportcited by 32
  9. 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methodscited by 30
  10. 2025V-Thinker: Interactive Thinking with Imagescited by 26
  11. 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
  12. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 25
38 more
  1. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  2. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 22
  3. 2025UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texturecited by 22
  4. 2025Latent Implicit Visual Reasoningcited by 18
  5. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 18
  6. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
  7. 2025Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Followingcited by 15
  8. 2025Towards Cross-View Point Correspondence in Vision-Language Modelscited by 14
  9. 2026DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editingcited by 14
  10. 2025Mantis: A Versatile Vision-Language-Action Model with Disentangled Visual Foresightcited by 13
  11. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 13
  12. 2026OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligencecited by 12
  13. 2026DM0: An Embodied-Native Vision-Language-Action Model towards Physical AIcited by 12
  14. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMscited by 11
  15. 2026Vero: An Open RL Recipe for General Visual Reasoningcited by 11
  16. 2026PersonaVLM: Long-Term Personalized Multimodal LLMscited by 11
  17. 2025InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Modelscited by 10
  18. 2026Dual Latent Memory for Visual Multi-agent Systemcited by 10
  19. 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Modelcited by 10
  20. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluationcited by 9
  21. 2025Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculationcited by 8
  22. 2025EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecturecited by 8
  23. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discoverycited by 8
  24. 2026VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?cited by 8
  25. 2026HiMemVLN: Enhancing Reliability of Open-Source Zero-Shot Vision-and-Language Navigation with Hierarchical Memory Systemcited by 8
  26. 2026ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understandingcited by 8
  27. 2026Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmarkcited by 8
  28. 2025Faithful-First Reasoning, Planning, and Acting for Multimodal LLMscited by 7
  29. 2026SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMscited by 7
  30. 2025IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Promptingcited by 6
  31. 2025DeliveryBench: Can Agents Earn Profit in Real World?cited by 6
  32. 2025Masking Teacher and Reinforcing Student for Distilling Vision-Language Modelscited by 6
  33. 2026Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policycited by 6
  34. 2026Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Modelscited by 6
  35. 2026Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuningcited by 6
  36. 2026Benchmark Everything Everywhere All at Oncecited by 6
  37. 2026EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understandingcited by 6
  38. 2026LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Taskscited by 5

50 most cited of 244 All 244 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token
  2. 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Models
  3. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models
  4. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
  5. 2023Pushing Boundaries: Exploring Zero Shot Object Classification with Large Multimodal Models
  6. 2024iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models
  7. 2024LLaVA-KD: A Framework of Distilling Multimodal Large Language Models
  8. 2023An Empirical Study of Scaling Instruct-Tuned Large Multimodal Models