mlsanityData to 7 Oct 2026

A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs

arXiv 2606.04596 ↗First posted 3 Jun 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) are increasingly used for video understanding, yet their reliability under multi-video inputs remains poorly understood. We study positional bias in multi-video summarization, where the quality of a per-video summary can change with the video's input slot even when the underlying content is unchanged. We construct a benchmark from ActivityNet and News videos, covering Cooking, Domestic, Leisure, and News settings with two- and four-video inputs. We evaluate nine open-source and proprietary MLLMs and measure position effects with three complementary metrics: Coverage, Directional Positional Bias (DPB), and Middle-Edge Gap (MEG). Our results show that positional effects are domain- and model-dependent: signed directional bias can be small even when middle positions underperform, and increasing visual or generation budget does not uniformly remove the imbalance. We further analyze prompt-level mitigation methods. Together, the results show that multi-video summarization remains sensitive to input protocol and position, motivating more robust order-invariant multimodal systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 30 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

15 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding24 Mar

Other references · 14

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  5. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  6. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  7. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  8. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  9. 2024Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encodingcited by 72
  10. 2019Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Lossescited by 22
  11. 2024On Positional Bias of Faithfulness for Long-form Summarizationcited by 22
  12. 2021Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluationcited by 21
2 more
  1. 2025Video-LevelGauge: Investigating Contextual Positional Bias in Large Video Language Modelscited by 6
  2. 2026QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answeringcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LVSum: A Benchmark for Timestamp-Aware Long Video Summarization
  2. 2026Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity
  3. 2026HAVEN: Hierarchically Aligned Multimodal Benchmark for Unified Video Understanding
  4. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Models
  5. 2026Multimodal Abstractive Summarization of Instructional Videos with Vision-Language Models
  6. 2025Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure
  7. 2026Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events
  8. 2026HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization