mlsanityData to 14 Oct 2026

Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models

arXiv 2410.19732 ↗First posted 25 Oct 2024cs.CL cs.CVcited by 42

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) excel in cross-model tasks but experience performance declines in long-context reasoning due to overreliance on textual information and reduced visual dependency. In this study, we empirically analyze LVLMs in long-context reasoning, revealing that increased context length leads to a higher dependence on language at the expense of visual dependency. To address this issue, we propose a novel training-free context pruning method that selectively removes less critical textual information. Our approach enhances visual dependency and reduces textual noise, thereby improving LVLM performance in long-context reasoning. We validate our method by constructing a long-context dataset, demonstrating its effectiveness across various LVLMs. Moreover, further analysis confirms the robustness of different token pruning strategies and preliminary explores scaling laws between pruning rates and context length.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 30 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

13 of 30 references.

References · 13

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  3. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  4. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  5. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  6. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  7. 2023SVIT: Scaling up Visual Instruction Tuningcited by 162
  8. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
  9. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
  10. 2023Thread of Thought Unraveling Chaotic Contextscited by 72
  11. 2024Contextual Position Encoding: Learning to Count What's Importantcited by 54
  12. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoningcited by 23
1 more
  1. 2024SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Informationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 21; 2026: 10.

42 papers.

Sort:

By area:

  1. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 37
  2. 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
  3. 2025AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understandingcited by 9
  4. 2025Contextualizing Search Queries In-Context Learning for Conversational Rewriting with LLMscited by 4
  5. 2025X-Ray-CoT: Interpretable Chest X-ray Diagnosis with Vision-Language Models via Chain-of-Thought Reasoningcited by 4
  6. 2025Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMscited by 3
  7. 2025Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMscited by 3
  8. 2026Enhancing Self-Correction in Large Language Models through Multi-Perspective Reflectioncited by 3
  9. 2026Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guaranteescited by 3
  10. 2024Large Vision-Language Models for Remote Sensing Visual Question Answeringcited by 2
  11. 2025Knowledge-Augmented Language Models Interpreting Structured Chest X-Ray Findingscited by 2
  12. 2024Temporal Contrastive Learning for Video Temporal Reasoning in Large Vision-Language Modelscited by 1
30 more
  1. 2025Dynamic Knowledge Integration for Enhanced Vision-Language Reasoningcited by 1
  2. 2025Evidence-Driven Marker Extraction for Social Media Suicide Risk Detectioncited by 1
  3. 2025Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagramscited by 1
  4. 2026Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rankcited by 1
  5. 2026Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalizationcited by 1
  6. 2024Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detectioncited by 0
  7. 2024Leveraging Large Language Models for Comparative Literature Summarization with Reflective Incremental Mechanismscited by 0
  8. 2024Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learningcited by 0
  9. 2024Reasoning-Aware Query-Focused Summarization over Multi-Table Datacited by 0
  10. 2024Dynamic Cross-Modal Alignment for Robust Semantic Location Predictioncited by 0
  11. 2024Selective State Space Memory for Large Vision-Language Modelscited by 0
  12. 2024Bridging Vision and Language: Modeling Causality and Temporality in Video Narrativescited by 0
  13. 2024Optimizing Vision-Language Interactions Through Decoder-Only Modelscited by 0
  14. 2024Leveraging Retrieval-Augmented Tags for Large Vision-Language Understanding in Complex Scenescited by 0
  15. 2025Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Modelscited by 0
  16. 2025Vision-Driven Prompt Optimization for Large Language Models in Multimodal Generative Taskscited by 0
  17. 2025Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Modelscited by 0
  18. 2025Cross-Domain Semantic Segmentation with Large Language Model-Assisted Descriptor Generationcited by 0
  19. 2025Multi-granular Training Strategies for Robust Multi-hop Reasoning Over Noisy and Heterogeneous Knowledge Sourcescited by 0
  20. 2025Elevating Visual Question Answering through Implicitly Learned Reasoning Pathways in LVLMscited by 0
  21. 2025Enhancing Contrastive Demonstration Selection with Semantic Diversity for Robust In-Context Machine Translationcited by 0
  22. 2025Revolutionizing Radiology Workflow with Factual and Efficient CXR Report Generationcited by 0
  23. 2025VisuCraft: Enhancing Large Vision-Language Models for Complex Visual-Guided Creative Content Generation via Structured Information Extractioncited by 0
  24. 2025VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosiscited by 0
  25. 2026AE-LLM: Adaptive Efficiency Optimization for Large Language Modelscited by 0
  26. 2026Avoiding Overthinking and Underthinking: Curriculum-Aware Budget Scheduling for LLMscited by 0
  27. 2026HierBias: Context-Conditioned Hierarchical Media Bias Detection with Multi-Task Type Classificationcited by 0
  28. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0
  29. 2026When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPOcited by 0
  30. 2026TerraAgent: Demonstration-Grounded Multi-Agent Reasoning for Long-Context Remote Sensing Visual Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts
  2. 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
  3. 2026Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models
  4. 2026SwiftVLM: Efficient Vision-Language Model Inference via Cross-Layer Token Bypass
  5. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  6. 2026CRISP: Pre-LLM Yet Text-Driven Visual Token Pruning for Efficient LVLM Inference
  7. 2025EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models
  8. 2025STAR: Stage-Wise Attention-Guided Token Reduction for Efficient Large Vision-Language Models Inference