mlsanityData to 14 Oct 2026

Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

arXiv 2605.00814 ↗First posted 1 May 2026cs.CV cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

While autoregressive Large Vision-Language Models (LVLMs) demonstrate remarkable proficiency in multimodal tasks, they face a "Visual Signal Dilution" phenomenon, where the accumulation of textual history expands the attention partition function, causing visual attention to decay inversely with generated sequence length. To counteract this, we propose Persistent Visual Memory (PVM), a lightweight learnable module designed to strengthen sustained, on-demand access to visual evidence. Integrated as a parallel branch alongside the Feed-Forward Network (FFN) in LVLMs, PVM establishes a distance-agnostic retrieval pathway that directly provides visual embeddings for enhanced visual perception, thereby structurally mitigating the signal suppression inherent to deep generation. Extensive experiments on Qwen3-VL models demonstrate that PVM brings notable improvements with negligible parameter overhead, delivering consistent average accuracy gains across both 4B and 8B scales, particularly in complex reasoning tasks that demand persistent visual perception. Furthermore, in-depth analysis reveals that PVM shows improved robustness in longer generations and accelerates internal prediction convergence.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 87 of its 94 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

87 of 94 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training28 Sep
  2. 2025ContextualLVLM-Agent: A Holistic Framework for Multi-Turn Visually-Grounded Dialogue and Complex Instruction Following21 Aug
  3. 2025CoMemo: LVLMs Need Image Context with Image Memory6 Jun
  4. 2024Pixtral 12B9 Oct
  5. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models21 Sep
  6. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models17 Jul
  7. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI24 Apr
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  9. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun
  12. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 75

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2025Qwen3 Technical Reportcited by 8,820
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2020Scaling Laws for Neural Language Modelscited by 6,586
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  12. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
38 more
  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  5. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  6. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  7. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  8. 2016A Diagram Is Worth A Dozen Imagescited by 804
  9. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  10. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  11. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  12. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  13. 2025Kimi-VL Technical Reportcited by 388
  14. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  15. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323
  16. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 303
  17. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  18. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 145
  19. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  20. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Modelscited by 116
  21. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 97
  22. 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 86
  23. 2024Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMscited by 80
  24. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  25. 2024DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMscited by 75
  26. 2025VGR: Visual Grounded Reasoningcited by 73
  27. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Modelscited by 72
  28. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decodingcited by 63
  29. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  30. 2024Interleaved-Modal Chain-of-Thoughtcited by 51
  31. 2025Do Language Models Use Their Depth Efficiently?cited by 50
  32. 2024Memory Consolidation Enables Long-Context Video Understandingcited by 45
  33. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 42
  34. 2024Autoregressive Models in Vision: A Surveycited by 39
  35. 2025Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracycited by 39
  36. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  37. 2024CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worldscited by 38
  38. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skillscited by 10
  2. 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 1
  3. 2026TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memorycited by 0
  4. 2026ReMAP: Restoring the Perceptual Cycle with Reasoning-Time Latent Visual Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Dual-Latent Memory Routing for Vision-Language Reasoning
  2. 2026Black-Box Continual Learning for Vision-Language Models
  3. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  4. 2025Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting
  5. 2025Towards General Continuous Memory for Vision-Language Models
  6. 2025Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval
  7. 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Models
  8. 2025VideoMem: Enhancing Ultra-Long Video Understanding via Adaptive Memory Management