mlsanityData to 14 Oct 2026

Can Linguistic Reasoning Vectors Enhance Multimodal Reasoning Ability?

arXiv 2609.31140 ↗First posted 25 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However, this multimodal scaling often degrades the language-side reasoning ability originally encoded in the base LLM. While the base LLM retains usable reasoning after scaling, the aligned VLM itself cannot reliably access this ability. Therefore, recovering the degraded reasoning capability in VLMs would benefit more from seeking help from the base LLM than from the VLM alone. Motivated by this, we propose LIFT (Language-side reasonIng Facilitation and Transfer), a lightweight vector-intervention method that transfers reasoning capability from the base LLM to the VLM without retraining the backbone. LIFT defines Reasoning Vectors as answer-token hidden-state differences between a Reasoner path with an explicit reasoning trace and a Solver path without it, and injects these vectors into language-side activations of the target VLM. LIFT further supports learnable vector adaptation while keeping the VLM backbone frozen. We evaluate LIFT on two VLMs across six reasoning benchmarks, comparing Reasoning Vectors extracted from the base LLM and from the aligned VLM under matched protocols. Results show that LLM-derived vectors consistently outperform VLM-derived vectors, confirming that the base LLM is a more effective source for recovering reasoning. LIFT partially recovers degraded reasoning through lightweight language-side interventions. Further analyses show that Reasoning Vectors influence intermediate reasoning behavior rather than merely altering final answers. The source code will be released soon.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 44 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

42 of 44 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging8 May
  2. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  4. 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verification19 Feb
  5. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
  6. 2025Virgo: A Preliminary Exploration on Reproducing o1-like MLLM3 Jan
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  8. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale6 Dec
  9. 2024Training-Free Mitigation of Language Reasoning Degradation After Multimodal Instruction Tuning4 Dec
  10. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  11. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
  12. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
3 more
  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  2. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  10. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  12. 2022Editing Models with Task Arithmeticcited by 1,120
15 more
  1. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  2. 2023Steering Language Models With Activation Engineeringcited by 922
  3. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  4. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  5. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  6. 2023Function Vectors in Large Language Modelscited by 308
  7. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  8. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
  9. 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 221
  10. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  11. 2025When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformerscited by 38
  12. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 16
  13. 2025Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learningcited by 10
  14. 2024Visual Question Decomposition on Multimodal Large Language Modelscited by 7
  15. 2025Controlling Multimodal LLMs via Reward-guided Decodingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
  2. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  3. 2025See, Think, Learn: A Self-Taught Multimodal Reasoner
  4. 2026Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
  5. 2025MMC: Iterative Refinement of VLM Reasoning via MCTS-based Multimodal Critique
  6. 2023Large Language Models are Visual Reasoning Coordinators
  7. 2024Enhancing Advanced Visual Reasoning Ability of Large Language Models
  8. 2025Training-Free Reasoning and Reflection in MLLMs