# Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models

arXiv: https://arxiv.org/abs/2410.19732 · HTML: https://mlsanity.com/paper/2410.19732/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 13 of 13)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [SVIT: Scaling up Visual Instruction Tuning](https://mlsanity.com/paper/2307.04087/) (2023)
- [LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning](https://mlsanity.com/paper/2401.01325/) (2024)
- [MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs](https://mlsanity.com/paper/2406.11833/) (2024)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [Contextual Position Encoding: Learning to Count What's Important](https://mlsanity.com/paper/2405.18719/) (2024)
- [Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning](https://mlsanity.com/paper/2408.17150/) (2024)
- [SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information](https://mlsanity.com/paper/2409.14083/) (2024)

## Built on by (most cited first, 30 of 42)

- [VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models](https://mlsanity.com/paper/2511.11007/) (2025)
- [Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding](https://mlsanity.com/paper/2511.22850/) (2025)
- [AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understanding](https://mlsanity.com/paper/2507.02591/) (2025)
- [Contextualizing Search Queries In-Context Learning for Conversational Rewriting with LLMs](https://mlsanity.com/paper/2502.15009/) (2025)
- [X-Ray-CoT: Interpretable Chest X-ray Diagnosis with Vision-Language Models via Chain-of-Thought Reasoning](https://mlsanity.com/paper/2508.12455/) (2025)
- [Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMs](https://mlsanity.com/paper/2503.08857/) (2025)
- [Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMs](https://mlsanity.com/paper/2507.16951/) (2025)
- [Enhancing Self-Correction in Large Language Models through Multi-Perspective Reflection](https://mlsanity.com/paper/2601.07780/) (2026)
- [Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guarantees](https://mlsanity.com/paper/2603.22355/) (2026)
- [Large Vision-Language Models for Remote Sensing Visual Question Answering](https://mlsanity.com/paper/2411.10857/) (2024)
- [Knowledge-Augmented Language Models Interpreting Structured Chest X-Ray Findings](https://mlsanity.com/paper/2505.01711/) (2025)
- [Temporal Contrastive Learning for Video Temporal Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2412.11391/) (2024)
- [Dynamic Knowledge Integration for Enhanced Vision-Language Reasoning](https://mlsanity.com/paper/2501.08597/) (2025)
- [Evidence-Driven Marker Extraction for Social Media Suicide Risk Detection](https://mlsanity.com/paper/2502.18823/) (2025)
- [Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagrams](https://mlsanity.com/paper/2508.16972/) (2025)
- [Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rank](https://mlsanity.com/paper/2604.09704/) (2026)
- [Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization](https://mlsanity.com/paper/2604.19857/) (2026)
- [Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection](https://mlsanity.com/paper/2412.00890/) (2024)
- [Leveraging Large Language Models for Comparative Literature Summarization with Reflective Incremental Mechanisms](https://mlsanity.com/paper/2412.02149/) (2024)
- [Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning](https://mlsanity.com/paper/2412.08955/) (2024)
- [Reasoning-Aware Query-Focused Summarization over Multi-Table Data](https://mlsanity.com/paper/2412.08970/) (2024)
- [Dynamic Cross-Modal Alignment for Robust Semantic Location Prediction](https://mlsanity.com/paper/2412.09870/) (2024)
- [Selective State Space Memory for Large Vision-Language Models](https://mlsanity.com/paper/2412.09875/) (2024)
- [Bridging Vision and Language: Modeling Causality and Temporality in Video Narratives](https://mlsanity.com/paper/2412.10720/) (2024)
- [Optimizing Vision-Language Interactions Through Decoder-Only Models](https://mlsanity.com/paper/2412.10758/) (2024)
- [Leveraging Retrieval-Augmented Tags for Large Vision-Language Understanding in Complex Scenes](https://mlsanity.com/paper/2412.11396/) (2024)
- [Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Models](https://mlsanity.com/paper/2501.00917/) (2025)
- [Vision-Driven Prompt Optimization for Large Language Models in Multimodal Generative Tasks](https://mlsanity.com/paper/2501.02527/) (2025)
- [Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models](https://mlsanity.com/paper/2501.15571/) (2025)
- [Cross-Domain Semantic Segmentation with Large Language Model-Assisted Descriptor Generation](https://mlsanity.com/paper/2501.16467/) (2025)

## Abstract (arXiv)

Large Vision-Language Models (LVLMs) excel in cross-model tasks but experience performance declines in long-context reasoning due to overreliance on textual information and reduced visual dependency. In this study, we empirically analyze LVLMs in long-context reasoning, revealing that increased context length leads to a higher dependence on language at the expense of visual dependency. To address this issue, we propose a novel training-free context pruning method that selectively removes less critical textual information. Our approach enhances visual dependency and reduces textual noise, thereby improving LVLM performance in long-context reasoning. We validate our method by constructing a long-context dataset, demonstrating its effectiveness across various LVLMs. Moreover, further analysis confirms the robustness of different token pruning strategies and preliminary explores scaling laws between pruning rates and context length.