# Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding

arXiv: https://arxiv.org/abs/2607.28516 · HTML: https://mlsanity.com/paper/2607.28516/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 61 references that are in the map.
- First posted on arXiv 30 Jul 2026.

## Builds on (most cited first, 30 of 55)

- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic](https://mlsanity.com/paper/2306.15195/) (2023)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models](https://mlsanity.com/paper/2311.17043/) (2023)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding](https://mlsanity.com/paper/2410.17434/) (2024)
- [LongVILA: Scaling Long-Context Visual Language Models for Long Videos](https://mlsanity.com/paper/2408.10188/) (2024)
- [MLVU: Benchmarking Multi-task Long Video Understanding](https://mlsanity.com/paper/2406.04264/) (2024)
- [Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding](https://mlsanity.com/paper/2409.14485/) (2024)
- [Multi-Modal Hallucination Control by Visual Information Grounding](https://mlsanity.com/paper/2403.14003/) (2024)
- [VISA: Reasoning Video Object Segmentation via Large Language Models](https://mlsanity.com/paper/2407.11325/) (2024)
- [TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning](https://mlsanity.com/paper/2410.19702/) (2024)
- [GroundingGPT:Language Enhanced Multi-modal Grounding Model](https://mlsanity.com/paper/2401.06071/) (2024)
- [MIRAGE: The Illusion of Visual Understanding](https://mlsanity.com/paper/2603.21687/) (2026)
- [LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling](https://mlsanity.com/paper/2511.20785/) (2025)
- [VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos](https://mlsanity.com/paper/2411.04923/) (2024)
- [Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding](https://mlsanity.com/paper/2503.18478/) (2025)
- [Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers](https://mlsanity.com/paper/2503.11579/) (2025)

## Built on by (most cited first, 2 of 2)

- [PREM: Prefix-Steered Recurrent Memory for Long-Video Understanding](https://mlsanity.com/paper/2609.23601/) (2026)
- [TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models](https://mlsanity.com/paper/2609.33414/) (2026)

## Abstract (arXiv)

Long-video understanding commonly compresses videos into a small set of frames or visual tokens for answer generation. Existing compact pipelines focus on retaining relevant visual content as explicit evidence. Yet making evidence available does not ensure that complementary cues across moments are integrated for answering. Our key idea is to organize selected frames into query-relevant cross-frame evidence before generation. We formulate this post-selection stage as a latent evidence interface and instantiate it with GenEvA (Generative Latent Evidence Aggregation), a distribution-guided latent evidence aggregation framework. Specifically, GenEvA uses a query-conditioned evidence distribution to focus aggregation on relevant frames, forming compact cross-frame latent evidence from their frame-specific information. Since cross-frame integration is not always needed, the same distribution determines whether to insert this latent complement. Across four benchmarks and two Video-MLLM backbones, GenEvA consistently improves matched-frame baselines. At 8 frames, it raises the four-benchmark LLaVA-Video average by +5.2 points and Qwen2.5-VL accuracy on LVBench by +10.1 points. These gains require only 0.11%--0.40% average video-token overhead; analyses further show task-aware allocation and benefits from Adaptive Evidence Invocation.