mlsanityData to 14 Oct 2026

FineVision: Open Data Is All You Need

arXiv 2510.17269 ↗First posted 20 Oct 2025cs.CV cs.AIcited by 53

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The advancement of vision-language models (VLMs) is hampered by a fragmented landscape of inconsistent and contaminated public datasets. We introduce FineVision, a meticulously collected, curated, and unified corpus of 24 million samples - the largest open resource of its kind. We unify more than 200 sources into 185 subsets via a semi-automated, human-in-the-loop pipeline: automation performs bulk ingestion and schema mapping, while reviewers audit mappings and spot-check outputs to verify faithful consumption of annotations, appropriate formatting and diversity, and safety; issues trigger targeted fixes and re-runs. The workflow further applies rigorous de-duplication within and across sources and decontamination against 66 public benchmarks. FineVision also encompasses agentic/GUI tasks with a unified action space; reviewers validate schemas and inspect a sample of trajectories to confirm executable fidelity. Models trained on FineVision consistently outperform those trained on existing open mixtures across a broad evaluation suite, underscoring the benefits of scale, data hygiene, and balanced automation with human oversight. We release the corpus and curation tools to accelerate data-centric VLM research.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 127 of its 182 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

127 of 182 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models20 Jan
  2. 2024Building and better understanding vision-language models: insights and future directions22 Aug
  3. 2024SPARK: Multi-Vision Sensor Perception and Reasoning Benchmark for Large-scale Vision-Language Models22 Aug
  4. 2024MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models24 Jul
  5. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine11 Jul
  6. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models25 Jun
  7. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences16 Jun
  8. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models22 May
  9. 2024What matters when building vision-language models?3 May
  10. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models1 Mar
  11. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models18 Feb

Other references · 116

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  6. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  7. 2019Towards VQA Models That Can Readcited by 1,718
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  9. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  11. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
38 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  2. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  3. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  4. 2016A Diagram Is Worth A Dozen Imagescited by 804
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  6. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  7. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  8. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  9. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  10. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  11. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  12. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  13. 2021InfographicVQAcited by 437
  14. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  15. 2021OCR-free Document Understanding Transformercited by 407
  16. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  17. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  18. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 378
  19. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  20. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  21. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  22. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
  23. 2019Scene Text Visual Question Answeringcited by 331
  24. 2019PlotQA: Reasoning over Scientific Plotscited by 321
  25. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  26. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  27. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
  28. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 311
  29. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 309
  30. 2022Visual Spatial Reasoningcited by 293
  31. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  32. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  33. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  34. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267
  35. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 256
  36. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  37. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
  38. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237

50 most cited of 116 All 116 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 43.

53 papers.

Sort:

By area:

  1. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  2. 2026STEP3-VL-10B Technical Reportcited by 32
  3. 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methodscited by 30
  4. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 27
  5. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 19
  6. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 18
  7. 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 12
  8. 2026Vero: An Open RL Recipe for General Visual Reasoningcited by 11
  9. 2025OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoningcited by 10
  10. 2025InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Modelscited by 10
  11. 2026Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Modelcited by 10
  12. 2025EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecturecited by 8
38 more
  1. 2026Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoderscited by 7
  2. 2026Self-Supervised Visual On-Policy Distillationcited by 6
  3. 2025ConsistCompose: Unified Multimodal Layout Control for Image Compositioncited by 5
  4. 2026PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignmentcited by 5
  5. 2026Kwai Keye-VL-2.0 Technical Reportcited by 5
  6. 2026InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoningcited by 5
  7. 2026WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmarkcited by 4
  8. 2026Towards Predictive, Aligned, and Scalable Robot Learningcited by 3
  9. 2025Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Modelscited by 2
  10. 2026BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillationcited by 2
  11. 2026ZAYA1-VL-8B Technical Reportcited by 2
  12. 202620/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alonecited by 2
  13. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Modelscited by 2
  14. 2026Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Makingcited by 2
  15. 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to Systemcited by 2
  16. 2025jina-vlm: Small Multilingual Vision Language Modelcited by 1
  17. 2025Differences That Matter: Auditing Models for Capability Gap Discovery and Rectificationcited by 1
  18. 2026Linear Model Merging Unlocks Simple and Scalable Multimodal Data Mixture Optimizationcited by 1
  19. 2026Kelix Technical Reportcited by 1
  20. 2026OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Modelscited by 1
  21. 2026AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolutioncited by 1
  22. 2026See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RLcited by 1
  23. 2026MidTool: Mid-training Data Synthesis for Agentic Tool Usecited by 1
  24. 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 1
  25. 2026MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoningcited by 1
  26. 2025INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Modelscited by 0
  27. 2026FineViT: Progressively Unlocking Fine-Grained Perception with Dense Recaptionscited by 0
  28. 2026Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Modelscited by 0
  29. 2026Zamba2-VL Technical Reportcited by 0
  30. 2026Mind the Heads: Topological Representation Alignment for Multimodal LLMscited by 0
  31. 2026Bridging Video Understanding and Generation in a Unified Frameworkcited by 0
  32. 2026The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoningcited by 0
  33. 2026LookME: Lookup-Based Multimodal Embeddings for Layer Injection in Vision-Language Modelscited by 0
  34. 2026Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steeringcited by 0
  35. 2026When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoningcited by 0
  36. 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
  37. 2026DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answeringcited by 0
  38. 2026VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Modelscited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding
  2. 2025Data Metabolism: An Efficient Data Design Schema For Vision Language Model
  3. 2023Measuring Progress in Fine-grained Vision-and-Language Understanding
  4. 2026Scaling Vision-Language Models Is Not Enough to Mitigate Bias
  5. 2026FineGen: A VLM-based Multi-Agent Framework for Fine-Grained Image-Text Dataset Construction
  6. 2024VHELM: A Holistic Evaluation of Vision Language Models
  7. 2026Understanding the Fine-Grained Knowledge Capabilities of Vision-Language Models
  8. 2026JAMMEval: A Refined Collection of Japanese Benchmarks for Reliable VLM Evaluation