Visual Saliency Steering Distillation for Multimodal Chain-of-Thought Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal chain-of-thought (CoT) reasoning integrates visual and textual cues through step-by-step inference. In small models with limited token budgets, modality-interaction fusion often suppresses tiny cross-modal differences. In particular, multimodal CoT often struggles when different images pair with identical text or different texts pair with an identical image, making such inputs nearly indistinguishable after fusion. This study proposes Visual Saliency Steering Distillation (VSSD). VSSD leverages the attention maps of multimodal large language models to generate perturbed images that capture task-sensitive feature directions, and then applies singular value decomposition to extract dominant steering vectors to guide inter-layer distillation. Experiments on ScienceQA and M3CoT demonstrate that VSSD improves rationale generation and answer inference. The code is available at https://github.com/BGWH123/VSSD.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 18 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
17 of 18 references.
Closest ancestors · 9
Same direction, newest first
- 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought26 May
- 2024Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Sampling16 May
- 2023Boosting the Power of Small Multimodal Reasoning Models to Match Larger Models with Self-Consistency Training23 Nov
- 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models25 Oct
- 2023Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework24 Jul
- 2023IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models24 May
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models8 Mar
- 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
Other references · 8
- 2023Visual Instruction Tuningcited by 7,792
- 2024GPT-4o System Cardcited by 4,467
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
- 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
- 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
- 2023Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Modelscited by 100
- 2022Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Taggingcited by 14
- 2025Vision-aware Multimodal Prompt Tuning for Uploadable Multi-source Few-shot Domain Adaptationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
- 2026Distilling Visual Reasoning into Text Space
- 2026Beyond Static Visual Tokens: Structured Sequential Visual Chain-of-Thought Reasoning
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
- 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
- 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought
- 2026OPLD: On-Policy Latent Distillation for Multimodal Reasoning
- 2024Interleaved-Modal Chain-of-Thought