# Video Evidence Indexing: Learning Where to Look from Video Previews for Token-Budgeted Long-Video Question Answering

arXiv: https://arxiv.org/abs/2610.00757 · HTML: https://mlsanity.com/paper/2610.00757/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 47 of its 57 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 30 of 47)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding](https://mlsanity.com/paper/2501.13106/) (2025)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026)
- [Adaptive Keyframe Sampling for Long Video Understanding](https://mlsanity.com/paper/2502.21271/) (2025)
- [Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning](https://mlsanity.com/paper/2508.04416/) (2025)
- [Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs](https://mlsanity.com/paper/2506.22139/) (2025)
- [FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting](https://mlsanity.com/paper/2509.24304/) (2025)
- [FOCUS: Efficient Keyframe Selection for Long Video Understanding](https://mlsanity.com/paper/2510.27280/) (2025)
- [Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding](https://mlsanity.com/paper/2512.05774/) (2025)
- [Video-MTR: Reinforced Multi-Turn Reasoning for Long Video Understanding](https://mlsanity.com/paper/2508.20478/) (2025)
- [VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning](https://mlsanity.com/paper/2512.22315/) (2025)
- [VideoMem: Enhancing Ultra-Long Video Understanding via Adaptive Memory Management](https://mlsanity.com/paper/2512.04540/) (2025)
- [LensWalk: Agentic Video Understanding by Planning How You See in Videos](https://mlsanity.com/paper/2603.24558/) (2026)
- [Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior](https://mlsanity.com/paper/2512.06866/) (2025)
- [Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-in](https://mlsanity.com/paper/2512.14273/) (2025)
- [Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders](https://mlsanity.com/paper/2603.06569/) (2026)
- [AdaCodec: A Predictive Visual Code for Video MLLMs](https://mlsanity.com/paper/2606.02569/) (2026)
- [Unified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retention](https://mlsanity.com/paper/2603.21957/) (2026)
- [ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning](https://mlsanity.com/paper/2605.20342/) (2026)
- [RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models](https://mlsanity.com/paper/2607.24447/) (2026)
- [Self-Supervised Visual On-Policy Distillation](https://mlsanity.com/paper/2608.14144/) (2026)
- [Small Vision-Language Models are Smart Compressors for Long Video Understanding](https://mlsanity.com/paper/2604.08120/) (2026)

## Abstract (arXiv)

Long-video question answering is limited by the high cost of visual tokens and by the fixed context width of current VLMs. A long-video question may require broad temporal coverage, but the answer is often supported by only a compact set of moments. To locate these moments efficiently, we propose token-budgeted Video Evidence Indexing (VEI): given a dense low-resolution Video Preview, the model constructs a compact high-resolution Evidence Set for final reasoning. We treat VEI as a policy that must jointly solve textitevidence localization, which finds question-relevant moments, and textitbudget planning, which decides where to spend the limited high-resolution frame budget. We implement this idea with an inference pipeline: the Video Preview provides cheap global coverage, Video Evidence Indexing constructs the Evidence Set, and Answer Generation combines both inputs for final VQA. To address missing frame-level supervision, we adopt privileged self-distillation, where an answer-aware teacher guides the normal test-time policy on student-generated indexing traces. We explore previews at 1, 6, 12, and 24 visual tokens per frame, training a single policy that supports all four resolutions. Experiments show that Video Evidence Indexing improves accuracy under limited visual budgets, and self-distillation further improves both QA accuracy and temporal evidence localization.