StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answering
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Knowledge-based Visual Question Answering (KVQA) requires models to ground entities in images and reason over factual knowledge. Recent work has introduced its implicit-knowledge variant, IK-KVQA, where a multimodal large language model (MLLM) is the sole knowledge source and answers are produced without external retrieval. Existing IK-KVQA approaches, however, are typically trained with answer-only supervision: reasoning remains implicit, justifications are often weak or inconsistent, and generalization after standard supervised fine-tuning (SFT) can be brittle. We propose StaR-KVQA, a framework that equips IK-KVQA with dual-path structured reasoning traces - symbolic relation paths over text and vision together with path-grounded natural-language explanations - to provide a stronger inductive bias than generic answer-only supervision. These traces act as modality-aware scaffolds that guide the model toward relevant entities and attributes, offering more structure than generic chain-of-thought supervision while not constraining reasoning to any single fixed path. With a single open-source MLLM, StaR-KVQA constructs and selects traces to build an offline trace-enriched dataset and then performs structure-aware self-distillation; no external retrievers, verifiers, or curated knowledge bases are used, and inference is a single autoregressive pass. Across benchmarks, StaR-KVQA consistently improves both answer accuracy and the transparency of intermediate reasoning, achieving up to +11.3% higher answer accuracy on OK-VQA over the strongest baseline.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 56 of its 61 references that are in the map. First posted on arXiv 8 Oct 2025.
Builds on
56 of 61 references.
Closest ancestors · 3
Same direction, newest first
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
Other references · 53
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
38 more
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2015VQA: Visual Question Answeringcited by 2,506
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2025Gemma 3 Technical Reportcited by 1,824
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
- 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
- 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
- 2023DataComp: In search of the next generation of multimodal datasetscited by 644
- 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
- 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 339
- 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 328
- 2024What matters when building vision-language models?cited by 326
- 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
- 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 301
- 2018Bilinear Attention Networkscited by 280
- 2016FVQA: Fact-based Visual Question Answeringcited by 181
- 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
- 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 126
- 2020KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQAcited by 125
- 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 96
- 2024Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMscited by 80
- 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 79
- 2023Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Promptingcited by 62
- 2024EchoSight: Advancing Visual-Language Models with Wiki Knowledgecited by 62
- 2020Relational Message Passing for Knowledge Graph Completioncited by 61
- 2022REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answeringcited by 59
- 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 53
- 2022Subgraph Neighboring Relations Infomax for Inductive Link Prediction on Knowledge Graphscited by 27
- 2024RoRA-VLM: Robust Retrieval-Augmented Vision Language Modelscited by 23
- 2022VLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledgecited by 20
- 2024Modality-Aware Integration with Large Language Models for Knowledge-based Visual Question Answeringcited by 16
- 2025M2-Reasoning: Empowering MLLMs with Unified General and Spatial Reasoningcited by 16
- 2023A Survey on Temporal Knowledge Graph Completion: Taxonomy, Progress, and Prospectscited by 14
- 2023Prompt Tuning on Graph-augmented Low-resource Text Classificationcited by 13
- 2023S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learningcited by 10
- 2023Towards More Faithful Natural Language Explanation Using Multi-Level Contrastive Learning in VQAcited by 5
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting
- 2026MaS-VQA: A Mask-and-Select Framework for Knowledge-Based Visual Question Answering
- 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
- 2024Disentangling Knowledge-based and Visual Reasoning by Question Decomposition in KB-VQA
- 2022A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA
- 2024Find The Gap: Knowledge Base Reasoning For Visual Question Answering
- 2024Knowledge Condensation and Reasoning for Knowledge-based VQA
- 2024Precision Empowers, Excess Distracts: Visual Question Answering With Dynamically Infused Knowledge In Language Models