mlsanityData to 7 Oct 2026

Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning

arXiv 2608.01635 ↗First posted 3 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Despite the progress of multimodal large language models (MLLMs), they continue to exhibit deficiencies in visual perception. Following visual instruction tuning, internal MLLM representations rapidly deviate from their original semantic states during inference, causing severe information degradation. While existing methods attempt to leverage external vision foundation models (VFMs) to align internal representations, we find that direct alignment with VFMs enhances visual semantics but fails to mitigate representation deviation. To address this, we propose Spatial-Spectral Visual Anchor Learning (SSVAL). The core of SSVAL is Visual Anchor Prompt Injection (VAPI), which introduces prompts that absorb rich knowledge from external VFMs during training, enabling them to serve as stable visual anchors that mitigate representation deviation during inference. Additionally, we incorporate auxiliary spatial and frequency-domain representation alignment losses to provide complementary vision-specific supervision at intermediate LLM layers. Extensive experiments demonstrate that SSVAL significantly outperforms existing methods. Code are available on our hrefhttps://msls38.github.io/SSVAL/project page.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 45 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

31 of 45 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Explore How to Inject Beneficial Noise in MLLMs17 Nov
  2. 2025Visual Representation Alignment for Multimodal Large Language Models9 Sep
  3. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders28 Aug
  4. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs11 Jan
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  8. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models13 Nov
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  10. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  6. 2022Visual Prompt Tuningcited by 1,674
  7. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  8. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  10. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  11. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  12. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
8 more
  1. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  2. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  3. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters Morecited by 149
  4. 2024BRAVE: Broadening the visual encoding of vision-language modelscited by 68
  5. 2024Instruct Large Language Models to Drive like Humanscited by 11
  6. 2025Multimodal Continual Learning with MLLMs from Multi-scenario Perspectivescited by 4
  7. 2025Class-Aware Prototype Learning with Negative Contrast for Test-Time Adaptation of Vision-Language Modelscited by 2
  8. 2025Mitigating Long-Tail Bias in HOI Detection via Adaptive Diversity Cachecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Why Vision Fails as a Universal Bridge: Rectifying Modality Asynchrony in Multilingual MLLMs
  2. 2026VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
  3. 2025Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models
  4. 2025Towards Long-window Anchoring in Vision-Language Model Distillation
  5. 2026Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models
  6. 2025BASIC: Boosting Visual Alignment with Intrinsic Refined Embeddings in Multimodal Large Language Models
  7. 2023Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models
  8. 2024Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model