# Towards Sparse Video Understanding and Reasoning

arXiv: https://arxiv.org/abs/2602.13602 · HTML: https://mlsanity.com/paper/2602.13602/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 35 of its 84 references that are in the map.
- First posted on arXiv 14 Feb 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 35)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2504.20073/) (2025)
- [GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest](https://mlsanity.com/paper/2307.03601/) (2023)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models](https://mlsanity.com/paper/2408.04840/) (2024)
- [ReFT: Reasoning with Reinforced Fine-Tuning](https://mlsanity.com/paper/2401.08967/) (2024)
- [PG-Video-LLaVA: Pixel Grounding Large Video-Language Models](https://mlsanity.com/paper/2311.13435/) (2023)
- [LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos](https://mlsanity.com/paper/2312.05269/) (2023)
- [Memory Consolidation Enables Long-Context Video Understanding](https://mlsanity.com/paper/2402.05861/) (2024)
- [Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptions](https://mlsanity.com/paper/2304.04227/) (2023)
- [FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting](https://mlsanity.com/paper/2509.24304/) (2025)
- [FOCUS: Efficient Keyframe Selection for Long Video Understanding](https://mlsanity.com/paper/2510.27280/) (2025)
- [Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding](https://mlsanity.com/paper/2512.05774/) (2025)
- [FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding](https://mlsanity.com/paper/2504.17447/) (2025)
- [FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2509.24008/) (2025)
- [CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion](https://mlsanity.com/paper/2402.05889/) (2024)
- [ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding](https://mlsanity.com/paper/2506.01274/) (2025)
- [Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge](https://mlsanity.com/paper/2402.16050/) (2024)
- [Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critique](https://mlsanity.com/paper/2503.17363/) (2025)

## Built on by (most cited first, 2 of 2)

- [TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding](https://mlsanity.com/paper/2608.22516/) (2026)
- [Agentic Video Understanding: A Survey](https://mlsanity.com/paper/2609.31713/) (2026)

## Abstract (arXiv)

We present revise (underlineReasoning with underlineVideo underlineSparsity), a multi-round agent for video question answering (VQA). Instead of uniformly sampling frames, revise selects a small set of informative frames, maintains a summary-as-state across rounds, and stops early when confident. It supports proprietary vision-language models (VLMs) in a ``plug-and-play'' setting and enables reinforcement fine-tuning for open-source models. For fine-tuning, we introduce EAGER (Evidence-Adjusted Gain for Efficient Reasoning), an annotation-free reward with three terms: (1) Confidence gain: after new frames are added, we reward the increase in the log-odds gap between the correct option and the strongest alternative; (2) Summary sufficiency: at answer time we re-ask using only the last committed summary and reward success; (3) Correct-and-early stop: answering correctly within a small turn budget is rewarded. Across multiple VQA benchmarks, revise improves accuracy while reducing frames, rounds, and prompt tokens, demonstrating practical sparse video reasoning.