FlipVQA: Scaling Multi-modal Instruction Tuning via Textbook-to-Knowledge Synthesis
Not on the map. Why?
Abstract (arXiv)
Textbooks are among the richest repositories of human-verified reasoning knowledge, yet their complex layouts contain multi-column typesetting, cross-page question answer separation, and interleaved figures, make automated extraction of structured QA and VQA pairs extremely challenging. Existing alternatives either synthesize data from scratch, which lacks authentic problem contexts, or rely on costly expert annotation that cannot scale. We propose FlipVQA-Miner, an automated pipeline that resolves long-range logical dependencies and cross-page discontinuities in OCR-parsed documents, recovering coherent question--answer--figure associations even when answers reside in separate companion volumes. A subsequent multi-stage curation pipeline transforms these raw extractions into AI-ready supervision signals. Using FlipVQA-Miner, we construct FlipVQA-83K, comprising 83K QA and VQA pairs spanning 11 academic disciplines, at a 50× cost saving compared to manual annotation while maintaining high structural fidelity (F1 > 0.96). Models fine-tuned on FlipVQA-83K demonstrate significantly improved reasoning ability and cross-domain generalization, establishing a scalable paradigm for human-knowledge-grounded data curation. Our dataset and the complete data generating and curating methods can be found in https://github.com/OpenDCAI/DataFlow-VQA .
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 39 references that are in the map. First posted on arXiv 20 Nov 2025.
Builds on
15 of 39 references.
References · 15
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
- 2025Humanity's Last Examcited by 467
- 2023Nougat: Neural Optical Understanding for Academic Documentscited by 240
- 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
3 more
- 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 108
- 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 80
- 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Sciencecited by 15
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level Evaluation
- 2026A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR
- 2024TextSquare: Scaling up Text-Centric Visual Instruction Tuning
- 2022Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contexts
- 2025LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
- 2025Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality
- 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale
- 2024Enhancing textual textbook question answering with large language models and retrieval augmented generation