Data-Efficient Curation for Multimodal Reasoning under Fixed Training Protocols
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We study data curation for multimodal reasoning in a fixed-protocol fine-tuning regime, where the base model, optimizer, training schedule, and evaluation pipeline are held constant and the main degree of freedom is the training data. Using the NeurIPS 2025 Data Curation for Vision--Language Reasoning (DCVLR) challenge as a controlled testbed, we analyze how source-dataset alignment, model-relative difficulty, dataset size, diversity heuristics, and rewritten synthetic mixtures affect downstream reasoning accuracy. Among the tested interventions, difficulty filtering on an aligned source corpus provides the strongest gains at matched scale. The effect is not explained only by LiveXivTQA weighting: a per-benchmark decomposition shows that much of the improvement over random sampling comes from OlympiadBench, the largest non-LiveXivTQA benchmark. Qwen-derived difficulty scores also transfer to some additional model families, though the benefit is architecture-dependent. In contrast, increasing dataset size beyond roughly 1k aligned examples mainly reduces run-to-run variance under the fixed recipe, while the diversity and rewritten CoSyn mixtures we tested do not improve over the difficulty-filtered baseline. These results provide a scoped empirical recipe for data-constrained multimodal reasoning fine-tuning, rather than a universal claim about data selection across all training regimes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 3 of its 14 references that are in the map. First posted on arXiv 16 Jan 2026.
Builds on
3 of 14 references.
References · 3
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
- 2024LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Contentcited by 17
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
- 2025Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions
- 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe
- 2026Reasoning Quality Emerges Early: Data Curation for Reasoning Models
- 2026MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning
- 2026DataComp-VLM: Improved Open Datasets for Vision-Language Models
- 2026Dual Tuning for Reasoning Efficacy-Driven Data Curation in Multimodal LLM Training
- 2026Balanced Thinking: Improving Chain of Thought Training in Vision Language Models