MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
The rapid advancement of long-context vision language models (LCVLMs) has led to a significant expansion of their context windows. However, an extended context window does not guarantee the effective utilization of the context, posing a critical challenge for real-world applications. Current evaluations of such long-context faithfulness in multimodal settings remain limited to short contexts. To bridge this gap, we introduce MMLongCite, the first benchmark evaluating the faithfulness of LCVLMs via multimodal citation generation. MMLongCite features 2,280 examples across 8 tasks and diverse modalities (image, video, interleaved), with context lengths scaled from 16K to 128K tokens. To test spatial localization capabilities of LCVLMs, we also introduce MMLongCite-HR, evaluating fine-grained visual grounding amidst dense pixel spaces. Through extensive benchmarking of cutting-edge LCVLMs, we provide a systematic analysis of current multimodal citation capabilities. Our results reveal a significant discrepancy between answer correctness and citation faithfulness. We also conduct attention pattern investigations and in-depth error analyses to reveal the underlying phenomena of failures in LCVLMs. MMLongCite establishes a rigorous foundation for diagnosing and advancing the faithfulness of LCVLMs. We hope our findings provide meaningful insights to drive further improvements in the long-context capabilities of LCVLMs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 56 references that are in the map. First posted on arXiv 15 Oct 2025.
Builds on
16 of 56 references.
Closest ancestors · 1
Same direction, newest first
Other references · 15
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
- 2025Kimi-VL Technical Reportcited by 388
- 2025MiMo-VL Technical Reportcited by 115
- 2024MileBench: Benchmarking MLLMs in Long Contextcited by 88
- 2023A Survey of Large Language Models Attributioncited by 71
- 2024Why Does the Effective Context Length of LLMs Fall Short?cited by 63
- 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughlycited by 29
- 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
- 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 22
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Long Context Transfer from Language to Vision
- 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks
- 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
- 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models
- 2026Scaling Vision-Language Models Is Not Enough to Mitigate Bias
- 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
- 2025Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation
- 2024Visual Context Window Extension: A New Perspective for Long Video Understanding