# Elevating Visual Question Answering through Implicitly Learned Reasoning Pathways in LVLMs

arXiv: https://arxiv.org/abs/2503.14674 · HTML: https://mlsanity.com/paper/2503.14674/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 9 of 9)

- [A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges](https://mlsanity.com/paper/2501.02189/) (2025)
- [Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models](https://mlsanity.com/paper/2410.19732/) (2024)
- [EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models](https://mlsanity.com/paper/2502.04424/) (2025)
- [Self-Improved Learning for Scalable Neural Combinatorial Optimization](https://mlsanity.com/paper/2403.19561/) (2024)
- [Visual question answering: from early developments to recent advances -- a survey](https://mlsanity.com/paper/2501.03939/) (2025)
- [Diffusion Model with Representation Alignment for Protein Inverse Folding](https://mlsanity.com/paper/2412.09380/) (2024)
- [Socratic Questioning: Learn to Self-guide Multimodal Reasoning in the Wild](https://mlsanity.com/paper/2501.02964/) (2025)
- [Self-Improving SLAM in Dynamic Environments: Learning When to Mask](https://mlsanity.com/paper/2210.08350/) (2022)
- [Enhancing Embodied Object Detection through Language-Image Pre-training and Implicit Object Memory](https://mlsanity.com/paper/2402.03721/) (2024)

## Abstract (arXiv)

Large Vision-Language Models (LVLMs) have shown remarkable progress in various multimodal tasks, yet they often struggle with complex visual reasoning that requires multi-step inference. To address this limitation, we propose MF-SQ-LLaVA, a novel approach that enhances LVLMs by enabling implicit self-questioning through end-to-end training. Our method involves augmenting visual question answering datasets with reasoning chains consisting of sub-question and answer pairs, and training the LVLM with a multi-task loss that encourages the generation and answering of these intermediate steps, as well as the prediction of the final answer. We conduct extensive experiments on the ScienceQA and VQAv2 datasets, demonstrating that MF-SQ-LLaVA significantly outperforms existing state-of-the-art models, including the base LLaVA and the original SQ-LLaVA. Ablation studies further validate the contribution of each component of our approach, and human evaluation confirms the improved accuracy and coherence of the reasoning process enabled by our method.