Can Linguistic Reasoning Vectors Enhance Multimodal Reasoning Ability?
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However, this multimodal scaling often degrades the language-side reasoning ability originally encoded in the base LLM. While the base LLM retains usable reasoning after scaling, the aligned VLM itself cannot reliably access this ability. Therefore, recovering the degraded reasoning capability in VLMs would benefit more from seeking help from the base LLM than from the VLM alone. Motivated by this, we propose LIFT (Language-side reasonIng Facilitation and Transfer), a lightweight vector-intervention method that transfers reasoning capability from the base LLM to the VLM without retraining the backbone. LIFT defines Reasoning Vectors as answer-token hidden-state differences between a Reasoner path with an explicit reasoning trace and a Solver path without it, and injects these vectors into language-side activations of the target VLM. LIFT further supports learnable vector adaptation while keeping the VLM backbone frozen. We evaluate LIFT on two VLMs across six reasoning benchmarks, comparing Reasoning Vectors extracted from the base LLM and from the aligned VLM under matched protocols. Results show that LLM-derived vectors consistently outperform VLM-derived vectors, confirming that the base LLM is a more effective source for recovering reasoning. LIFT partially recovers degraded reasoning through lightweight language-side interventions. Further analyses show that Reasoning Vectors influence intermediate reasoning behavior rather than merely altering final answers. The source code will be released soon.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 44 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
42 of 44 references.
Closest ancestors · 15
Same direction, newest first
- 2025Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging8 May
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verification19 Feb
- 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
- 2025Virgo: A Preliminary Exploration on Reproducing o1-like MLLM3 Jan
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
- 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale6 Dec
- 2024Training-Free Mitigation of Language Reasoning Degradation After Multimodal Instruction Tuning4 Dec
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
3 more
Other references · 27
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2022Editing Models with Task Arithmeticcited by 1,120
15 more
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
- 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
- 2023Function Vectors in Large Language Modelscited by 308
- 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
- 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
- 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 221
- 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
- 2025When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformerscited by 38
- 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 16
- 2025Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learningcited by 10
- 2024Visual Question Decomposition on Multimodal Large Language Modelscited by 7
- 2025Controlling Multimodal LLMs via Reward-guided Decodingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
- 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
- 2025See, Think, Learn: A Self-Taught Multimodal Reasoner
- 2026Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
- 2025MMC: Iterative Refinement of VLM Reasoning via MCTS-based Multimodal Critique
- 2023Large Language Models are Visual Reasoning Coordinators
- 2024Enhancing Advanced Visual Reasoning Ability of Large Language Models
- 2025Training-Free Reasoning and Reflection in MLLMs